Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I'm running into an issue when reading a Glue Data Catalog data source in an Visual ETL AWS Glue job. An extra column is being added in called 'col40', which is not in the underlying file that was...
1
answers
0
votes
225
views
asked 3 months agolg...
hello, I am creating a dataframe consuming from a Glue Catalog table, this table has fields of type bigint, which can be null. It turns out that when this information is null, the dataframe ignores...
Accepted AnswerAWS Glue
1
answers
0
votes
177
views
asked 3 months agolg...
I was running glue job to process data from MariaDB inside VPC. Recently my glue job get "com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure" although it was running...
1
answers
2
votes
259
views
asked 3 months agolg...
Hello! According to the [documentation](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect-kinesis-home.html), it should be possible to write data to Kinesis from Glue...
2
answers
0
votes
1571
views
asked 3 months agolg...
I am working in the Glue ETL Visual Editor and I've started to encounter this error `QuotaExceededError - Failed to execute 'setItem' on 'Storage'`.
It is preventing me from even starting a Data...
0
answers
0
votes
130
views
asked 3 months agolg...
At every 30 minutes it saying "ICEBERG_VACUUM_MORE_RUNS_NEEDED: Removed 20000 files in this round of vacuum" but when I calculate the my table metadata size it didn't changed before and after the...
2
answers
0
votes
451
views
asked 3 months agolg...
I received this error - ResourceSetupError: Exception when listing images from AWS Glue. There are no logs because the duration of the job was 0s.
Why? =/
0
answers
0
votes
359
views
asked 3 months agolg...
I have a glue job (job_a) that starts through a Lambda. When a file is placed inside an S3 bucket, I am triggering a glue job (job_a) through Lambda. My requirement is, once this glue job (job_a), is...
1
answers
0
votes
379
views
asked 3 months agolg...
We are running into `No space left on device` errors in EMR Serverless for big jobs, even when setting driver / executor drive size to the maximum 200GB.
I tried to make the S3 shuffle storage...
1
answers
0
votes
246
views
asked 3 months agolg...
I am interested particularly in `%additional_python_modules` and I always get this error:
`UsageError: Line magic function `%additional_python_modules` not found.`
The same error is thrown when I...
2
answers
0
votes
157
views
asked 3 months agolg...
Hi,
we have a situation where an application running in a k8 environment of a different account have to access the athena and the glue data catalog in a different account.
since these two accounts...
1
answers
0
votes
216
views
asked 3 months agolg...
Similar to RedShift or Snowflake tables is there a way to perform UPSERT for RDS DBs or non RS/SF DB/tables using Glue Visual?
I know Spark Dataframe through JDBS connections only support Insert /...
1
answers
0
votes
124
views
asked 3 months agolg...