Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I'm trying to learn how to use this.
Not sure what the issue is behind the scenes, but I have 3 simple CSV files that I uploaded to S3.
I'm creating a test ETL pipeline with those three CSV files,...
3
answers
0
votes
144
views
asked 3 months agolg...
I am running a PoC around integrating the Glue lineage into the [DataHub](https://datahubproject.io/). I have based my research on this set of AWS blog posts...
1
answers
0
votes
557
views
asked 3 months agolg...
I have Security Lake enabled with my org level Cloud Trail. Events are coming into the Cloud Trail Management table, `amazon_security_lake_table_us_west_2_cloud_trail_mgmt_1_0`, in the underlying...
2
answers
0
votes
143
views
asked 3 months agolg...
We have a requirement where we need to register our Avro schema in Glue schema registry from a service running in my onprem cluster (outside AWS ). We have provisioned AWS Glue schema registry for...
1
answers
0
votes
201
views
asked 3 months agolg...
Hi, I am using AWS glue studio to read from a DDB table with direct DDB connection. So far my visual diagram has two nodes:
1. Source DDB table node -> Here preview takes 5 minutes for only 2 rows of...
1
answers
0
votes
259
views
asked 3 months agolg...
I have 2 AWS accounts, Account "A" contain AWS Redshift and Account "B" has external data that crawler from S3.
## What I have done
#### Account A
1. Attached spectrum role to Redshift
![spectrum...
1
answers
0
votes
425
views
asked 3 months agolg...
Is it possible to wildcard the include path for a MongoDB crawler. I've tried a number of different options similar to the options available for JDBC and other relational database connections, but...
1
answers
0
votes
136
views
asked 3 months agolg...
I receive a file from external vendor. The file is in ***.dat*** format. Once the file arrives into my S3 bucket, I have to trigger a AWS Glue job to read the file and load into my Redshift table. I...
2
answers
0
votes
212
views
asked 3 months agolg...
My dataframe has 2 columns - name and age. If there is name Manish with 2 rows one with age 16 and another with age 23 , will AWS data quality fail both, pass both or one fail one pass. for below...
1
answers
0
votes
225
views
asked 3 months agolg...
I have a glue job that transforms data from glue table. And I encounter the following error. It does not occur for every run of the job.
I have looked at a few documentarians, it seems to be coming...
1
answers
0
votes
358
views
asked 3 months agolg...
Hello
I am using Glue Pyspark to handle ETL, but when I tried running script with bookmark, I found out that if one script handles more than one table and one of them doesn't have changes or...
2
answers
0
votes
378
views
asked 3 months agolg...
When I try and add a new BigQuery connection as a sink for glue I am getting the following error:
InvalidInputException: jdbcEnforceSsl: is not defined in the schema and the schema does not allow...
1
answers
0
votes
176
views
asked 3 months agolg...