Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
We have a requirement where we need to register our Avro schema in Glue schema registry from a service running in my onprem cluster (outside AWS ). We have provisioned AWS Glue schema registry for...
1
answers
0
votes
188
views
asked 3 months agolg...
Hi, I am using AWS glue studio to read from a DDB table with direct DDB connection. So far my visual diagram has two nodes:
1. Source DDB table node -> Here preview takes 5 minutes for only 2 rows of...
1
answers
0
votes
229
views
asked 3 months agolg...
I have 2 AWS accounts, Account "A" contain AWS Redshift and Account "B" has external data that crawler from S3.
## What I have done
#### Account A
1. Attached spectrum role to Redshift
![spectrum...
1
answers
0
votes
356
views
asked 3 months agolg...
Is it possible to wildcard the include path for a MongoDB crawler. I've tried a number of different options similar to the options available for JDBC and other relational database connections, but...
1
answers
0
votes
126
views
asked 3 months agolg...
I receive a file from external vendor. The file is in ***.dat*** format. Once the file arrives into my S3 bucket, I have to trigger a AWS Glue job to read the file and load into my Redshift table. I...
2
answers
0
votes
199
views
asked 3 months agolg...
My dataframe has 2 columns - name and age. If there is name Manish with 2 rows one with age 16 and another with age 23 , will AWS data quality fail both, pass both or one fail one pass. for below...
1
answers
0
votes
209
views
asked 3 months agolg...
I have a glue job that transforms data from glue table. And I encounter the following error. It does not occur for every run of the job.
I have looked at a few documentarians, it seems to be coming...
1
answers
0
votes
346
views
asked 3 months agolg...
Hello
I am using Glue Pyspark to handle ETL, but when I tried running script with bookmark, I found out that if one script handles more than one table and one of them doesn't have changes or...
2
answers
0
votes
361
views
asked 3 months agolg...
When I try and add a new BigQuery connection as a sink for glue I am getting the following error:
InvalidInputException: jdbcEnforceSsl: is not defined in the schema and the schema does not allow...
1
answers
0
votes
166
views
asked 3 months agolg...
I have a Kinesis stream that is persisting (inserting) data to an iceberg table, via a Glue streaming job. I'm following the glue streaming pattern as published...
2
answers
0
votes
1157
views
asked 3 months agolg...
I tried AWS Glue data quality dynamic rules in my AWS Glue pipeline. I wrote below rule
RowCount > avg(last(3))
Then I processed 3 csv files with 1000,10000 and 100 rows. Then in 4th run I again...
1
answers
0
votes
187
views
asked 3 months agolg...
Glue service with SSL enabled document db connection is always timing out. when i disable cert ,my connection is working fine.
I am not sure where to pass the SSL cert when i create a documentdb...
4
answers
0
votes
475
views
asked 3 months agolg...