Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Currently, I am using AWS Glue to extract data from Mongo DB and push alll data to json file in S3 service. I use **create_dynamic_frame** function to extract data from MongoDB and use...
1
answers
0
votes
58
views
asked 3 days agolg...
I'm creating a role in AWS Glue to read CSV files from an S3 bucket. I'm granting full access to S3, but I can't seem to avoid this error. I contacted support, and they suggested increasing the usage...
0
answers
0
votes
48
views
asked 6 days agolg...
Hello, so the context is:
We have a DMS service that sends data from Oracle to s3, after that a job is run to create the raw stage, and after that raw to trusted. The problem is that they changed...
0
answers
0
votes
34
views
asked 7 days agolg...
I am trying to create two DynamicFrames based on a column that is a boolean. I have tried
`dyf.split_rows({'mybool': {'=': 'true'}}, 'is_true', 'is_not_true')`
`dyf.split_rows({'mybool': {'=':...
2
answers
0
votes
65
views
asked 8 days agolg...
AWS Glue Job Errorlg...
Im trying to convert CSV files in S3 to Parquet in another S3 bucket. So first I read the CSV files using a crawler, load the data into a Table, and then use a Job to convert from the Table to S3 in...
0
answers
0
votes
117
views
asked 8 days agolg...
Is there a way to use AWS SDK or some python script to get all the job names that failed on specific job queue on AWS Batch? After extracting them, I would like to upload this to AWS S3 or RDS so that...
1
answers
0
votes
348
views
asked 9 days agolg...
I'm having the same issue. Data is stored in below format in s3 as JSON array with partitions
S3 path - s3://fleet-fuelcard-data-import-dev/lambda/fuelsoft-morgan/660306/2024/Apr/03-Apr-2024.json....
1
answers
0
votes
111
views
asked 14 days agolg...
In AWS Glue jobs, within the Targets node, I am unable to see the data types such as struct, array or map while changing the schema. Does AWS Glue not support these data types?
1
answers
0
votes
159
views
asked 16 days agolg...
I've successfully set up AWS Glue with an RDS database serving as the data source and a Snowflake database as the data target. In this setup, I've configured AWS Glue crawlers to catalog the metadata...
0
answers
0
votes
247
views
asked 17 days agolg...
Hi,
When I try to export (expdp) a database schema in Oracle RDS using SQL Developer, I get the error below related to dump file size. This seems to be caused by default FILESIZE parameter in AWS...
1
answers
0
votes
270
views
asked 19 days agolg...
Hello, I am relatively new to Glue and encountering some challenges with Glue ETL.
Our setup involves a datalake that retrieves data from a backend database as its source. This datalake is...
1
answers
0
votes
262
views
asked 22 days agolg...
Hello,
I have parquets files in S3 that i parse using Glue Crawler and query in Athena. I found that some files have two columns "x" and "y" that have a type **int64** while other files have them as...
1
answers
0
votes
219
views
asked 23 days agolg...