Explore how you can quickly prepare for, respond to, and recover from security events. Learn more.
All Content tagged with AWS Data Pipeline
AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specified intervals.
Content language: English
Select up to 5 tags to filter
Sort by most recent
92 results
I have a use-case where I have to read DynamoDB table data and convert this into csv file and write into a s3 bucket using AWS lambda function. DynamoDB tables data is bit low around 2-3 MB so I want ...
I followed [this AWS blog post](https://aws.amazon.com/blogs/aws/introducing-amazon-kinesis-data-analytics-studio-quickly-interact-with-streaming-data-using-sql-python-or-scala/) to create KDA app, an...
Hi There,
I'm sourcing data from dynamo db and residing into s3 bucket using AWS data pipeline. I have been running this pipeline once in a week to get up-to date the records from dynamo db table. E...
**Transfer data from RDS MySQL to S3 bucket
**
I set up a data pipeline to transfer data from RDS Mysql to S3 Bucket, where before thet I set up the RDS mysql database with username and password. (IA...
asked 3 years ago
Hi,
I have a use case where I am fetching data on certain items (unique itemID) multiple times a day (identified by day_BatchTime) and storing them on a dyanmoDB. My composite primary key consists o...
Hello
I am working on serverless application, and i was looking for something handle the frontend part and I found Honeycode since its native and its codeless.
So is it possible to my Honeycode app...
Hi,
I have a problem in that I make heavy use of EMRs, and I orchestrate their use with Data Pipeline - multiple daily runs are automated and EMRs are launched and terminated on conclusion.
However,...
I'm trying to create a data pipeline to export Dynamodb data to S3, but after following the online guide to the letter, the DataPipelineDefaultResourceRole isn't in the dropdown referred to above, the...
## Main problem
I understand that is no need to add Auto Scaling to an EMR cluster launched by Data Pipeline. Instead, we can specify the **capacity up-front** and it will be used for the duration of ...
Currently I use the [CreateExportTask](https://docs.aws.amazon.com/ko_kr/AmazonCloudWatchLogs/latest/APIReference/API_CreateExportTask.html) API to backup my log data.
The problem is, exported data o...
I'm confused about how staging of an S3Datanode is billed when done as part of a ShellCommandActivity with the 'stage' property set to true (i.e. I do not have CSV data and am not using a CopyActivity...
I have a few questions regarding data preparation for Forecast.
I have a dataset with about 3,000 item_id's, the data is recorded on weekdays only (no row for weekends/holidays), and the forecast h...