Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
I have created a Glue job to trigger it for S3 event. So I have below design
S3 Bucket ---> SQS ---> Lambda ---> Trigger Glue job from Lambda.
I am facing below error when multiple files are...
2
answers
0
votes
127
views
asked a month agolg...
Hello,
I work as a data engineer and business intelligence specialist for a fintech startup. We've entered into a new agreement with a supplier to provide a technological solution for managing their...
0
answers
0
votes
239
views
asked a month agolg...
Hello, I am relatively new to Glue and encountering some challenges with Glue ETL.
Our setup involves a datalake that retrieves data from a backend database as its source. This datalake is...
1
answers
0
votes
282
views
asked a month agolg...
Hello,
I have parquets files in S3 that i parse using Glue Crawler and query in Athena. I found that some files have two columns "x" and "y" that have a type **int64** while other files have them as...
1
answers
0
votes
234
views
asked a month agolg...
Unable to push Glue job to GitHub. Empty connections list is now allowed if connection is specified.lg...
Hi,
I am trying to Push the Glue job to GitHub repo. I have got added the access permissions to my role as specified in...
3
answers
0
votes
128
views
asked a month agolg...
I am using aws lakeformation workflow to create a data lake following [this](https://docs.aws.amazon.com/lake-formation/latest/dg/getting-started-tutorial-jdbc.html) guide. everything is setup as...
0
answers
0
votes
113
views
asked a month agolg...
Trying to connect to redshift database in python shell jobs. Tried with packages like psycopg2, redshift connector and pg. All of them gave similar error, hence I'm assuming problem is to establish...
1
answers
0
votes
187
views
asked a month agolg...
I've imported a dataset of JSON objects that all have consistent schema. Glue crawler finished successfully and created a table. I can select * with a limit in Athena, but when I select all rows I get...
1
answers
1
votes
308
views
asked a month agolg...
trying out AWS glue for the first time. i want to read some data from s3 and push it to some database in a database within aws. what resources , do i need to create/provision for AWS glue, if doing...
1
answers
0
votes
94
views
asked a month agolg...
Hello, I am attempting to update a glue job default argument within a lambda function.
So far the best I have been able to find is to use the boto3 get_job and update_job functions with the desired...
1
answers
0
votes
173
views
asked a month agolg...
In our ETL process we are building out a pipeline where someones job is to take input files (ex. csv) and map the columns to existing column names. After the mapping is complete a glue workflow will...
0
answers
0
votes
181
views
asked a month agolg...
Hi!
I am trying to use DataZone to share an Iceberg table from the Glue Data Catalog with another AWS account.
I have created the table with Athena in the source account like this:
```sql
CREATE...
0
answers
0
votes
103
views
asked a month agolg...