Questions tagged with AWS Glue
AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development.
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I have a glue job that is supposed to read from DynamoDB table of size 1.4GB, process it and write to Redshift. The job always fails with:
**'An error occurred while calling o181.pyWriteDynamicFrame....
Hi,
I am trying to read a csv file and then write to Delta file in S3 in AWS Glue notebook. Getting error:
Caused by: java.lang.ClassNotFoundException: delta.DefaultSource
I am using below :
from...
I have a AWS glue table with one partition named dt, i can add data in my s3, using Athena via this glue table and can also query it.
But I am not able to query data using redshift query editor.
I...
I'm trying to find out if Trino on EMR supports access controls maintained in Lake Formation. My catalog is AWS Glue. I couldn't find any documentation on Lake Formation or EMR side that would talk...
asked 2 months ago
Hi everyone, I changed the KMS key in Glue Catalog setting. So I need to delete my tables, and then re-create them by running Crawlers. it seems that deleting and recreating tables causes the bookmark...
asked 2 months ago
Hey guys, I created ETL jobs with 'Change schema' nodes, renamed the source keys, and dropped the keys I didn't need.
When I edit these jobs, I see that the renames and the drop keys are not saved.
Do...
Hi everyone, I got stuck with something.
if bookmark is reset it means that all data will be scanned. how can I estimate the cost for that? let's explain and can you guide me with this following...
asked 2 months ago
I have set up default IAM role in the step -
"Admins: Grant access to AWS Glue and set a default IAM role."
But in a new glue job, this IAM role is not appearing by default.
What is missing? Is...
Hi everyone. I did one experiment and found out in Glue if we delete a table and re-create it by crawler it has effect on glue bookmark (for ETL jobs). it is like reset bookmark. am I correct?
and...
asked 2 months ago
I need to write a Log Insights query to find all of the job run ID's associated with a Glue job within a certain time frame and when I write what I believe is the correct query I come up with 0 of 0...
Hello there!
I am trying to create a kind of push_down_predicate from a sql_query using the conceptions from [https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-pushdown.html]().
I...
I am trying to create an aws glue rotine which consum an database table from datacatalog and an csv, in this way join this table based on two columns (on from each table). After that i added an regex...