Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Questions tagged with AWS Glue

AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development.

Content language: English

Filter questions
Select tags to filter
Sort by
Sort by most recent
Filter Questions by:

Browse through the questions and answers listed below or filter and sort to narrow down your results.

2045 results

AWS Glue crawler creation fails with "Account <ID> is denied...

I’m unable to create an AWS Glue crawler in my AWS account, and the error suggests the problem may be related to account-level Glue provisioning, not IAM. Environment Region: us-east-1 Not part of an...
AWS Glue
1
answers
0
votes
88
views
asked 3 months ago

Athena: HIVE_FILESYSTEM_ERROR

I have a s3 bucket with nested partition, first is the account_id and another is dt=yyyy-mm-dd-hh-mm, of 10 min interval, for a day there can be 24*6 partitions. There can be missing partition in betw...
Amazon AthenaAWS GlueAmazon Simple Storage ServiceAmazon S3 Glacier
1
answers
0
votes
1K
views
asked 3 months ago

Glue Workflow: Clone/Copy a Workflow

How can I clone, or make a copy, or a Workflow? I can't seem to find any docs around this, and my attempts with the [CLI](https://docs.aws.amazon.com/cli/latest/reference/glue/create-workflow.html) a...
AWS Glue
1
answers
0
votes
90
views
asked 3 months ago

Glue Workflow: Delete one job node

I have a Workflow with one Trigger. I assign two Jobs to the Trigger, and then realise one of the Jobs was added in error. The [docs](https://docs.aws.amazon.com/glue/latest/dg/creating_running_workf...
Accepted AnswerAWS Glue
2
answers
0
votes
68
views
asked 3 months ago

AWS Glue Table Optimizer

Hello, We are interested in using the Table Optimizer feature provided by AWS Glue, but we noticed that its not available in us-west-1 where our buckets and resources are present. Relevant AWS docume...
AWS GlueAnalytics
1
answers
0
votes
44
views
asked 3 months ago

Modified data through custom script is updated but when join...

I have written a custom script to transform a column, data transformed successfully and I can verify that in data preview in aws glue studio, acft_tail_num -> 8911(earlier this was N8911 ) when I joi...
AWS GlueExtract Transform & Load Data
2
answers
0
votes
71
views
asked 3 months ago

How to increase logging level for spark workers in Glue 5.0?

Spark workers generate a lot of logs and most of the information is not required on day-to-day basic. I would like to pay less for logs pushed and at the same time to have control over the logs verbos...
AWS Glue
1
answers
0
votes
117
views
asked 3 months ago

Inconsistent AI metadata generation behavior in AWS DataZone...

Hi team, I’m observing a strange behavior with AI-generated metadata in AWS DataZone. We have two tables with similar structures (Both Glue Tables with no specific difference). For one of them, the ...
Amazon DataZoneAWS GlueAmazon Bedrock
1
answers
0
votes
96
views
asked 3 months ago

Glue streaming job throws FileDownloadException from S3 loca...

Hi, We have an AWS Glue streaming job that runs 24x7 and consumes messages from MSK. Yesterday, the job started failing during batch processing with the following exception: ``` ERROR GlueLogger: B...
AWS Glue
1
answers
0
votes
114
views
asked 3 months ago

Use of Table optimization in Glue for Iceberg Tables

Hi Everyone! I have an iceberg table with +999 files in data folder and metadata as well. I have enabled Optimization option in Glue in order to compact small files into larger, as well as snapshots r...
Accepted AnswerAWS GlueAmazon Athena
1
answers
0
votes
185
views
asked 3 months ago

Error Conection Glue-Redshsift

I'm trying to set up the Glue and Redshift connection. I created several endpoints for S3, Redshift, KMS, and Secret Manager, but I'm getting this error. I don't understand where this VPC value was co...
AWS GlueServerlessAmazon Redshift
1
answers
0
votes
137
views
asked 3 months ago

Creating Iceberg table with Pyiceberg

Hi! I am trying to create iceberg table on AWS from an ECS container using pyiceberg. The role for the ECS has access to s3 bucket and KMS policy. We use this role for other tasks and it can write wit...
AnalyticsAmazon Simple Storage ServiceContainersAWS Glue
1
answers
0
votes
154
views
asked 3 months ago
  • 1
  • 2
  • 3
  • 4
  • 5
  • •••
  • 171
  • Page size
    12 / page