Questions tagged with Amazon EMR
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
One of my dev team members, asking to share the emr spark artifacts s3 location for building a Java application. I referred this doc...
Accepted AnswerAmazon EMR
1
answers
1
votes
147
views
asked 4 months agolg...
We have a airflow setup runs the EMR jobs daily basis. I noticed an odd behavior that when I resubmit job for calculating the adhoc reports, spark application failed with below error, arguments seems...
Accepted AnswerAmazon EMR
1
answers
0
votes
164
views
asked 4 months agolg...
When I try to create a new workspace for an AWS EMR Studio in the AWS Console, I get a blank page and a Javascript error in the console
("Failed to execute 'mark' on 'Performance':...
0
answers
0
votes
98
views
asked 4 months agolg...
I am trying to have glue data catalog as the hive metastore, stood up the EMR(emr-6.15.0) with the following node classification config per AWS, and it always initialize a default glue catalog...
1
answers
0
votes
377
views
asked 4 months agolg...
So I define manually finishing using the RunJobFlow operator (https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html) `"KeepJobFlowAliveWhenNoSteps": True`. However, the cluster...
1
answers
0
votes
138
views
asked 4 months agolg...
I would like to know the log4j configuration to get container logs into more structured format like Json, so I can leverage another automation to parse the files and train some customization to filter...
2
answers
0
votes
324
views
asked 4 months agolg...
Hello,
I have upgraded the EMR from 6.14 to 6.15, and started seeing errors on the existing core node:
`org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException: IAMInstanceCredentialsProvider:...
0
answers
0
votes
92
views
asked 4 months agolg...
I am trying to connect to my documentDB trhough the spark-mongodb connector, but it looks like DocumentDB does not support Collstats. How disable the collstats command so i can do my transformations...
1
answers
0
votes
334
views
asked 4 months agolg...
How to add additional library i.e. databricks spark xml to a running EMR cluster and access it in Notebook
1
answers
0
votes
190
views
asked 5 months agolg...
I am using emr-6.12.0 and trying to set environment varibles which are stored in secret manager in bootstrap.sh file.
```
SECRET_NAME="/myapp/dev/secrets"
SECRETS_JSON=$(aws secretsmanager...
1
answers
0
votes
254
views
asked 5 months agolg...
I want my EMR cluster to be terminated automatically post an idle time.
I have configured 'Automatically terminate cluster after idle time' and set the idle time as '5 minutes' .
In my cluster i have...
1
answers
0
votes
204
views
asked 5 months agolg...
If my environment is full of Apache Hudi integrating with EMR and Lake Formation, I found out that Hudi environment is not very friendly to be used by Redshift nor Athena. There are many advanced...
1
answers
0
votes
408
views
asked 5 months agolg...