Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Facing the following error when using 2 jobs and 1 crawler:
Job 1: changing the schema and saving the csv file as parquet in S3.
Job 2: ETL Process
Crawler: Saving it in the AWs Glue Datacatalog...
1
answers
0
votes
610
views
asked 16 days agolg...
Currently, I am using AWS Glue to extract data from Mongo DB and push alll data to json file in S3 service. I use **create_dynamic_frame** function to extract data from MongoDB and use...
2
answers
0
votes
114
views
asked 17 days agolg...
I'm creating a role in AWS Glue to read CSV files from an S3 bucket. I'm granting full access to S3, but I can't seem to avoid this error. I contacted support, and they suggested increasing the usage...
0
answers
0
votes
63
views
asked 20 days agolg...
Hi,
I have a Glue which reads data from Database and database connection details like host, user name and password are stored in AWS secret manager. We have environment specific AWS secret manager...
1
answers
0
votes
60
views
asked 20 days agolg...
AWS Glue 4.0 support Apache Hudi 0.12.1 version. What steps can I follow to upgrade the version of Hudi to 0.14 in AWS Glue 4.0
2
answers
0
votes
80
views
asked 21 days agolg...
This was working before, as recently as a week or two ago but Athena now fails with "INVALID_PARAMETER_USAGE: Incorrect number of parameters: expected 207 but found 0." when the query has more than...
0
answers
0
votes
75
views
asked 21 days agolg...
I am trying to create two DynamicFrames based on a column that is a boolean. I have tried
`dyf.split_rows({'mybool': {'=': 'true'}}, 'is_true', 'is_not_true')`
`dyf.split_rows({'mybool': {'=':...
2
answers
0
votes
85
views
asked 21 days agolg...
I am writing this question after going through bunch of glue pricing documents. Essentially what I want to know is how glue divides visual job ETL components for pricing.
**Pipeline...
1
answers
0
votes
91
views
asked 21 days agolg...
AWS Glue Job Errorlg...
Im trying to convert CSV files in S3 to Parquet in another S3 bucket. So first I read the CSV files using a crawler, load the data into a Table, and then use a Job to convert from the Table to S3 in...
0
answers
0
votes
316
views
asked 21 days agolg...
I have a json file in s3 (sample below) in json lines format. I create a crawler in aws glue to read this file, which creates a table definition and produces a table schema as such ,
schema:
```
# ...
1
answers
0
votes
110
views
asked 22 days agolg...
I am setting up the Connection in AWS Glue, trying to connect to my dev db instance (AWS rds Aurora PostgreSQL), I double checked VPC, subnet and setup the inbound rule to allow incoming connections...
1
answers
0
votes
558
views
asked 24 days agolg...
Hello, can anyone give any advice on this.
I created the very simple test Glue job: Source - RDS Postgres, Destination - S3 bucket.
Run takes about 23 minuts and ends with timeout error.
In the log I...
3
answers
0
votes
609
views
asked 25 days agolg...