Questions tagged with Amazon EMR
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I am trying to create a EMR cluster with version 6.13.0 and spark installed on it with version Spark 3.4.1 But the service role attached gives error that it does not have EC2 permissions.
I tried...
3
answers
0
votes
589
views
asked 7 months agolg...
EMR upgrade with AL2lg...
I have a emr 5.29 cluster running spark and hive application and received notification that AL1 will be unsupported after Dec 23. I am working on applications migration to emr 5.36. Questions,
1. What...
Accepted AnswerAmazon EMR
1
answers
1
votes
233
views
asked 7 months agolg...
I am running catalog queries like spark.catalog.tables, or show databases or show tables. All these commands failing emr_instance_profile role is not authorized to perform: glue:CreateDatabase on...
2
answers
3
votes
272
views
asked 7 months agolg...
Would like to have information about Graviton instance support EMR studio schedule/plan/roadmap?
1
answers
0
votes
605
views
asked 7 months agolg...
I am drafting a yaml for simple EMR cluster. It will be loaded to stack and run.
But I am tried to launch my yml with servicerole without any attached policies and jobflowrole without any policies....
2
answers
0
votes
343
views
asked 7 months agolg...
After setting up a Hadoop cluster on AWS, I've noticed that when I attempt to stop the instances once I'm done with my work for the day and plan to resume the next day, the instances are automatically...
1
answers
0
votes
296
views
asked 7 months agolg...
I want to know if the option exist in EMR or automated way to kill the spark job that says initial job not accepted any resources. Some of my cluster is transient that run off batch hour without...
Accepted AnswerAmazon EMR
1
answers
1
votes
306
views
asked 7 months agolg...
Hi,
Lets say we have 1 TB file in s3 to process it using aws emr .
As per documentation it says we need to select hardware 3 times the size of input file . So for processing 1 TB file we need ec2...
2
answers
2
votes
1354
views
asked 7 months agolg...
I am running Flink jobs in the AWS EMR. I current as-is situation is,
1. The flink job logs are currently getting stored in HDFS and it grows over time.
2. Currently there is no log rolling...
1
answers
0
votes
268
views
asked 7 months agolg...
I am having difficulty working reading and writing complex types from spark to redshift. I am using the EMR-6.12 image. One point of confusion is that the docs...
1
answers
0
votes
491
views
asked 7 months agolg...
Hi all, I was wondering if someone has encountered the following and it this is a known problem (with hopefully a simple workaround).
We are using docker images from our private ECR repo in Spark...
1
answers
1
votes
431
views
asked 8 months agolg...
Suppose, while creating an EMR I have selected two instance types for primary and core fleet, one in us-east-1a and the second in us-east-1b.....Now, if the cluster got launched in us-east-1a and...
1
answers
1
votes
229
views
asked 8 months agolg...