Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I want to use AWS Glue Data Catalog as a metastore. I'm running an EMR Serverless job that inserts and updates data in a Delta Table. I've successfully populated Delta tables on my localhost...
2
answers
0
votes
145
views
asked 2 months agolg...
Hi, I'm implementing a case where either one column can be null, but not both in the same record. And implementing rule
(ColumnValues "col_1" = NULL) or (ColumnValues "col_2" = NULL)
I'm seeing below...
1
answers
0
votes
68
views
asked 2 months agolg...
Glue Job - S3 to S3lg...
Hi Team,
I am working on Glue to job to copy/move file from one bucket to another bucket. Could you please help me with your thoughts
1. Using Python how to copy/move the unzipped file to target...
0
answers
0
votes
87
views
asked 2 months agolg...
Following this post: https://repost.aws/knowledge-center/glue-reduce-cloudwatch-logs
I have created the following glue job:
```
from awsglue.context import GlueContext
from pyspark.context import...
1
answers
0
votes
131
views
asked 2 months agolg...
Hi,
I am testing Amazon DataZone features and therefore set up a domain together with another associated account.
I enabled the DataLake blueprint in both accounts. I have 2 projects (producer,...
2
answers
0
votes
136
views
asked 2 months agolg...
Hi,
I am runing a Spark ETL glue job and want to see how the executor's log through Spark UI. However, I didn't see the executor logs like stderr and stdout. Can you please advise how to find the...
2
answers
0
votes
134
views
asked 2 months agolg...
I can't use my personal root account to create Glue job or Glue Crawler
got such as
Failed to update job
[gluestudio-service.ap-southeast-1.amazonaws.com] createJob: AccessDeniedException: Account...
2
answers
0
votes
147
views
asked 2 months agolg...
My company is pushing to have everything "As Code" and I was wondering which tool is better for us.
Since we are not allowed to use the Glue interface to create ETLs, to configure the settings, and...
1
answers
0
votes
126
views
asked 2 months agolg...
Hello!
I am new to AWS Glue and I starting to create data monitoring rules in AWS Glue. I have tried multiple options with CustomSQL but can not seem to find the solution.
My problem: I want to check...
1
answers
0
votes
176
views
asked 2 months agolg...
Hello,
We have an AWS Glue Workflow that triggers a a lot of jobs, but eventually, out of the blue an error occurs in the workflow that doesn't generate any logs:
Workflow run metadata size...
Accepted AnswerAWS Glue
1
answers
0
votes
103
views
asked 2 months agolg...
Hi, basically what title says. Data connections test and crawlers work fine. I've tried enabling SSL, adding ''?enabledTLSProtocols=TLSv1.2' to connection string. Both did not help. CA on the rds is...
1
answers
1
votes
112
views
asked 2 months agolg...
I am trying to create a Glue job for migrating data from BigQuery to an S3 bucket. I have created a BigQuery connection in AWS Glue. The connection uses the Secret Id to reference a secret name in AWS...
1
answers
0
votes
129
views
asked 2 months agolg...