Questions tagged with AWS Data Pipeline
AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specified intervals.
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I am attempting to deploy a Flask (python3.8) application via AWS Elastic Beanstalk (EB). I was able to successfully deploy the application on a --single EC2 instance configuration in the public...
Hi all,
I saw this announcement today:
> Please note that Data Pipeline service is in maintenance mode and we are not planning to expand the service to new regions. We plan to remove console access...
Hello , I need to purge all data from dynamodb table except the last 1 year of data .
I do not have a TTL attribute set in the table, what is the best approach to proceed?
IT will cost a lot if I...
Hi All,
We have multiple sql servers installed on windows on prem environment, now plan is migrate those to AWS RDS.. We are planning to us below method.
1) Native import/Export using S3 and the...
I'm a beginner in tasks of date engineer.
I have a taks to create a data lakehouse and i'm tryding undestand how to do it using these tools: DMS, S3, Glue and Hudi.
I already created a simple data...
Hello! I am looking for an equivalent to this solution that MIcrosoft has flaunted called IDP intelligent data platform, it is governance + operations + analytics in one. they flaunt synapse with aml...
asked 2 years ago
I am trying to build a data pipeline for a data engineering project With the help of S3, Glue, Athena, etc., I am stuck when setting up glue crawler for indexing over data. Even I set up the role...
hi all,
I have a Dag that I schedule to run every quarterly using: **'schedule_interval='@quarterly'**, but for some reason it runs every day.
I don't see the run in managed airflow UI but the...
My aws account got hacked and my lightsail instace deleted- is there any recover option for that instance -please help
Now we got our account back but the instance is no more .And i don't have any...
Using AWS Pydeequ in databricks I am performing Data Quality checks. When I run this below mentioned code it provide only metrics results as my output (like Check_level, check_status, constraint,...
As a publisher of data set on AWS Data Exchange, we would like the data asset downloaded by customers on their server from AWS Data Exchange to be technically un-usable when they terminate the...
Would like to know what would be the way to migrate per user data currently hosted at on-prem vdi to AWS workspaces? Will this be a case to case migration since the on-prem VDI data might be only...