Questions tagged with AWS Data Pipeline
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
I have an EmrActivity step on the data pipeline that backups the dynamoDB and stores it in S3. I would like to use this functionality on a step function or something other than Data Pipeline. If...
1
answers
0
votes
303
views
asked a year agolg...
We have Microsoft SQL DB and we are taking backup of data in CSV format from MSSQL db and uploading to S3 bucket. I tried to query data using Athena and AWS API Gateway to build API. Athena looks do...
1
answers
0
votes
282
views
asked a year agolg...
I'm attempting to use AWS Data Pipeline to move a CSV file from my computer to AWS Data Lake as a parquet file. I'm unable to finad the exact template to select to migrate from my local...
0
answers
0
votes
88
views
asked a year agolg...
![.](/media/postImages/original/IMtFz5_gZCQ7e7aT9-QTcUuQ)
1
answers
0
votes
282
views
asked a year agolg...
Hello, when we see the AWS data pipeline console we see a message that AWS is planning to remove access to console by 04/03/2023. So we are checking how can we use the data pipelines from AWS CLI...
1
answers
0
votes
649
views
asked a year agolg...
I am transforming my table by adding new columns using SQL Query Transform in AWS Glue Job studio.
![visual diagram for transformation](/media/postImages/original/IMwcLXRM0iTROC0Uqb5lvOGg)
SQL...
1
answers
0
votes
943
views
asked a year agolg...
I want to run multiple python scripts to run one after another These are connected, what is the best way to do it ( want to use Ec2 Instance)
2
answers
0
votes
850
views
asked a year agolg...
I am looking for a way to share my Terraform-based modern data stack solution, which I have created using various AWS services, with other companies privately via AWS. However, in the future, we might...
1
answers
0
votes
330
views
asked a year agolg...
Where has the option of configuring request and response mapping template gone when creating a resolver for your data source? The way it was done in VTL?
0
answers
0
votes
106
views
asked a year agolg...
I am attempting to deploy a Flask (python3.8) application via AWS Elastic Beanstalk (EB). I was able to successfully deploy the application on a --single EC2 instance configuration in the public...
0
answers
0
votes
127
views
asked a year agolg...
Hi all,
I saw this announcement today:
> Please note that Data Pipeline service is in maintenance mode and we are not planning to expand the service to new regions. We plan to remove console access...
0
answers
0
votes
288
views
asked 2 years agolg...
Hello , I need to purge all data from dynamodb table except the last 1 year of data .
I do not have a TTL attribute set in the table, what is the best approach to proceed?
IT will cost a lot if I...
1
answers
0
votes
764
views
asked 2 years agolg...