Questions tagged with AWS Data Pipeline
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hello
I am working on serverless application, and i was looking for something handle the frontend part and I found Honeycode since its native and its codeless.
So is it possible to my Honeycode app...
1
answers
0
votes
515
views
asked 2 years agolg...
Hi,
I have a problem in that I make heavy use of EMRs, and I orchestrate their use with Data Pipeline - multiple daily runs are automated and EMRs are launched and terminated on conclusion.
However,...
1
answers
0
votes
371
views
asked 2 years agolg...
I'm trying to create a data pipeline to export Dynamodb data to S3, but after following the online guide to the letter, the DataPipelineDefaultResourceRole isn't in the dropdown referred to above, the...
1
answers
1
votes
573
views
asked 2 years agolg...
## Main problem
I understand that is no need to add Auto Scaling to an EMR cluster launched by Data Pipeline. Instead, we can specify the **capacity up-front** and it will be used for the duration of...
0
answers
0
votes
151
views
asked 2 years agolg...
Currently I use the [CreateExportTask](https://docs.aws.amazon.com/ko_kr/AmazonCloudWatchLogs/latest/APIReference/API_CreateExportTask.html) API to backup my log data.
The problem is, exported data...
1
answers
1
votes
1615
views
asked 2 years agolg...
I'm confused about how staging of an S3Datanode is billed when done as part of a ShellCommandActivity with the 'stage' property set to true (i.e. I do not have CSV data and am not using a...
0
answers
0
votes
219
views
asked 2 years agolg...
I have a few questions regarding data preparation for Forecast.
I have a dataset with about 3,000 item_id's, the data is recorded on weekdays only (no row for weekends/holidays), and the forecast...
0
answers
0
votes
113
views
asked 2 years agolg...
What's the best way to filter out duplicated records in a Glue ETL Job with bookmarking enabled?lg...
I have an etl pipeline that loads json data from a source bucket, runs an etl job with bookmarking enabled, and writes as parquet to a target bucket.
I'd like to ensure that the target bucket never...
1
answers
0
votes
5932
views
asked 2 years agolg...
I'm working on a step function state machine and can create lambdas in python and node to update an existing item in ddb. However, I can't seem to find any examples with service integrations AND...
1
answers
0
votes
667
views
asked 2 years agolg...
On the AWS EMR console, we are seeing AWS EMR 6.5.0 version being available.
However, EMR Documentation doesn't have any specific information on 6.5.0.
When will the documentation be updated based on...
1
answers
0
votes
434
views
asked 2 years agolg...
Hello,
Where can I find more details on AWS' approach around data models? This would include industry-specific data models AWS is fully invested in.
1
answers
0
votes
262
views
asked 2 years agolg...
We currently run into the Problem, that some data analytic workloads exceed the 15 minutes timeout of a lambda.
It is a multistep process with steps that are parallelizable and some that are not....
3
answers
0
votes
2748
views
asked 2 years agolg...