Skip to content

Questions tagged with AWS Glue

AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development.

Content language: English

Filter questions
Select tags to filter
Sort by
Sort by most recent
Filter Questions by:

Browse through the questions and answers listed below or filter and sort to narrow down your results.

2045 results
I’m unable to create an AWS Glue crawler in my AWS account, and the error suggests the problem may be related to account-level Glue provisioning, not IAM. Environment Region: us-east-1 Not part of an...
1
answers
0
votes
88
views
asked 3 months ago
I have a s3 bucket with nested partition, first is the account_id and another is dt=yyyy-mm-dd-hh-mm, of 10 min interval, for a day there can be 24*6 partitions. There can be missing partition in betw...
1
answers
0
votes
1K
views
asked 3 months ago
How can I clone, or make a copy, or a Workflow? I can't seem to find any docs around this, and my attempts with the [CLI](https://docs.aws.amazon.com/cli/latest/reference/glue/create-workflow.html) a...
1
answers
0
votes
90
views
asked 3 months ago
I have a Workflow with one Trigger. I assign two Jobs to the Trigger, and then realise one of the Jobs was added in error. The [docs](https://docs.aws.amazon.com/glue/latest/dg/creating_running_workf...
Accepted AnswerAWS Glue
2
answers
0
votes
68
views
asked 3 months ago
Hello, We are interested in using the Table Optimizer feature provided by AWS Glue, but we noticed that its not available in us-west-1 where our buckets and resources are present. Relevant AWS docume...
1
answers
0
votes
44
views
asked 3 months ago
I have written a custom script to transform a column, data transformed successfully and I can verify that in data preview in aws glue studio, acft_tail_num -> 8911(earlier this was N8911 ) when I joi...
2
answers
0
votes
71
views
asked 3 months ago
Spark workers generate a lot of logs and most of the information is not required on day-to-day basic. I would like to pay less for logs pushed and at the same time to have control over the logs verbos...
1
answers
0
votes
117
views
asked 3 months ago
Hi team, I’m observing a strange behavior with AI-generated metadata in AWS DataZone. We have two tables with similar structures (Both Glue Tables with no specific difference). For one of them, the ...
1
answers
0
votes
96
views
asked 3 months ago
Hi, We have an AWS Glue streaming job that runs 24x7 and consumes messages from MSK. Yesterday, the job started failing during batch processing with the following exception: ``` ERROR GlueLogger: B...
1
answers
0
votes
114
views
asked 3 months ago
Hi Everyone! I have an iceberg table with +999 files in data folder and metadata as well. I have enabled Optimization option in Glue in order to compact small files into larger, as well as snapshots r...
Accepted AnswerAWS GlueAmazon Athena
1
answers
0
votes
185
views
asked 3 months ago
I'm trying to set up the Glue and Redshift connection. I created several endpoints for S3, Redshift, KMS, and Secret Manager, but I'm getting this error. I don't understand where this VPC value was co...
1
answers
0
votes
137
views
asked 3 months ago
Hi! I am trying to create iceberg table on AWS from an ECS container using pyiceberg. The role for the ECS has access to s3 bucket and KMS policy. We use this role for other tasks and it can write wit...
1
answers
0
votes
154
views
asked 3 months ago
  • 1
  • 2
  • 3
  • 4
  • 5
  • •••
  • 171
  • Page size
    12 / page