Explore how you can quickly prepare for, respond to, and recover from security events. Learn more.
Questions tagged with AWS Data Pipeline
AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specified intervals.
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
89 results
I have to convert a pipeline to step function and I have this step which loads a dynamo db table with data from S3. I am not sure how to convert this to step function. Below is the step. Any idea how ...
Good afternoon.
I would like to know SE when we send a snapshoot of mysql 5.7 rds allocated storage size of 8tb.
When I restore from S3 to a new RDS instance, it can only consume the instance that has...
Hi there,
We are migrating from datapipeline to step function, and on datapipeline I am able to forward the log messages for individual steps to a log file, but this doesn't work on step function. A...
Hi All
I am trying to create a batch segment job and I have created the segment solution and while creating the batch job, I entered the link to input file on S3 and also added link to output folder ...
I want to read data from "Bentley Projectwise " and "MS Form" . Can anyone please suggest what are the best ways to connect and read data from Bentley Projectwise and MSForms?
Hi,
I have an EmrActivity step on the data pipeline that backups the dynamoDB and stores it in S3. I would like to use this functionality on a step function or something other than Data Pipeline. If ...
We have Microsoft SQL DB and we are taking backup of data in CSV format from MSSQL db and uploading to S3 bucket. I tried to query data using Athena and AWS API Gateway to build API. Athena looks do n...
I'm attempting to use AWS Data Pipeline to move a CSV file from my computer to AWS Data Lake as a parquet file. I'm unable to finad the exact template to select to migrate from my local computer.
ple...
Hello, when we see the AWS data pipeline console we see a message that AWS is planning to remove access to console by 04/03/2023. So we are checking how can we use the data pipelines from AWS CLI comm...
I am transforming my table by adding new columns using SQL Query Transform in AWS Glue Job studio.
SQL ali...
I want to run multiple python scripts to run one after another These are connected, what is the best way to do it ( want to use Ec2 Instance)