Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I've been trying to test out Iceberg tables with Amazon Redshift Spectrum and have come across a major issue.
Here is my setup:
1. I create an iceberg table via spark (emr 7.0) and insert data across...
0
answers
1
votes
7
views
asked 19 minutes agolg...
Hi,
I want to read a tar file from s3, uncompress it and load it to another s3 bucket using Glue job. But I am facing "fileobj must implement read".
obj=s3.getObject(bucketname,key)
objbuffer =...
1
answers
0
votes
47
views
asked 12 hours agolg...
In AWS Glue jobs, within the Targets node, I am unable to see the data types such as struct, array or map while changing the schema. Does AWS Glue not support these data types?
1
answers
0
votes
84
views
asked a day agolg...
I got the error when write glue dynamic data frame to redshift table .
The code I have used :-
my_conn_op={
"dbtable":"public.xxxx",
...
0
answers
0
votes
58
views
asked a day agolg...
Hi,
I've created Glue Crawler to determine data sctructure from XML file uploaded to s3, and write Table into Data Catalog.
I tried 2 approaches:
1. Use Glue default classifier - this is preferred...
1
answers
0
votes
53
views
asked 2 days agolg...
Hello.
I am trying to configure specific iam permission for an user. I need a permission for only read tables from existing Data Catalog.
So, I have configured this policiy:
```
{
"Version":...
0
answers
0
votes
87
views
asked 2 days agolg...
I've successfully set up AWS Glue with an RDS database serving as the data source and a Snowflake database as the data target. In this setup, I've configured AWS Glue crawlers to catalog the metadata...
0
answers
0
votes
102
views
asked 2 days agolg...
I am running a glue job with python script shell(version 3.9) and glue version is 3.0. I am passing 8 arguments to the glue job and accessing it using getResolvedOptions(args, options). One of the...
2
answers
0
votes
103
views
asked 5 days agolg...
Hi,
I have created a Glue job to trigger it for S3 event. So I have below design
S3 Bucket ---> SQS ---> Lambda ---> Trigger Glue job from Lambda.
I am facing below error when multiple files are...
2
answers
0
votes
69
views
asked 5 days agolg...
Hello,
I work as a data engineer and business intelligence specialist for a fintech startup. We've entered into a new agreement with a supplier to provide a technological solution for managing their...
0
answers
0
votes
181
views
asked 6 days agolg...
Hello, I am relatively new to Glue and encountering some challenges with Glue ETL.
Our setup involves a datalake that retrieves data from a backend database as its source. This datalake is...
1
answers
0
votes
173
views
asked 7 days agolg...
Hello,
I have parquets files in S3 that i parse using Glue Crawler and query in Athena. I found that some files have two columns "x" and "y" that have a type **int64** while other files have them as...
1
answers
0
votes
182
views
asked 8 days agolg...