Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi this question is regarding corrupt or malformed records in Glue ETL.
Spark DataFrames obviously have an option for indicated column for _corrupt_record when this happens and the entire record is...
1
answers
0
votes
182
views
asked 3 months agolg...
Hello, I would like to know if there is a way to query Iceberg tables (backed with S3 parquet files) cataloged within the AWS Glue Catalog using AWS Databrew. (maybe through Athena?).
Also, is it...
2
answers
0
votes
540
views
asked 3 months agolg...
Hi
Trying to craw connect logs create bad metadata with fields like this inside the table:
struct<connect\:Subtype:struct<ValueString:string>>
obvious running this struct inside athena result in a...
0
answers
0
votes
420
views
asked 3 months agolg...
Hi,
Have followed the below documentation to set up the Spark History server to see Spark UI Logs. Am able to run the container but not able to access the URL http://localhost:18080 .
docker run...
1
answers
0
votes
205
views
asked 3 months agolg...
We connected Timestream to Athena using the [Athena Timestream connector](https://docs.aws.amazon.com/athena/latest/ug/connectors-timestream.html). When running a federated query through Athena to...
2
answers
0
votes
744
views
asked 4 months agolg...
i got a mongodb atlas cluster outside aws. I want to use aws glue with my mongo db databases so i created a connection but im getting "InvalidInputException: Unable to resolve any valid connection". ...
1
answers
1
votes
216
views
asked 4 months agolg...
```
df = spark.read.parquet("s3://folder/")
df = df.withColumn('filename', input_file_name())
AmazonS3_node1697616892615 = DynamicFrame.fromDF(df, glueContext, "s3sparkread")
```
if this is the code...
1
answers
0
votes
272
views
asked 4 months agolg...
I'm trying to achieve data change capture using AWS Glue and don't want to use DMS. I'm trying to transfer data between two Oracle RDS instances which are in different AWS Account. Here I am trying to...
1
answers
0
votes
456
views
asked 4 months agolg...
I'm trying to achieve data change capture using AWS Glue and don't want to use DMS. I'm trying to transfer data between two Oracle RDS instances which are in different AWS Account. Here I am trying to...
1
answers
0
votes
443
views
asked 4 months agolg...
Hi all,
I'm recently introduced to AWS DataZone.
I'm have created 2 domain in a single account.
**df-test-1 and df-test-2**
Both domains do have data created by AWS Athena environment but I can...
1
answers
0
votes
590
views
asked 4 months agolg...
I tried to add through 2 ways
```
newdf = newdf.withColumn('filename2', input_file_name())
```
also tried
```
AmazonS3_node = glueContext.create_dynamic_frame.from_options(
...
1
answers
0
votes
370
views
asked 4 months agolg...
For a DeltaTarget, glue.create_crawler is not recognizing the parameter "CreateNativeDeltaTable"lg...
Unknown parameter in Targets.DeltaTargets[0]: "CreateNativeDeltaTable", must be one of: DeltaTables, ConnectionName, WriteManifest.
However, in the documentation it shows the parameter (see...
1
answers
0
votes
149
views
asked 4 months agolg...