Questions tagged with Amazon EMR
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
My customer is using AWS EMR and is storing all the Hive meta data on an external RDS instance, using MySQL 5.7.* And since MySQL 5.7 is running out of its lifecycle, we are pushing them to upgrade...
1
answers
1
votes
234
views
asked 5 months agolg...
Everyday a new emr cluster span up and terminated after completing the step job. Checking the cloudtrail, seems a Data Pipeline created it. I am not sure how to get more details like who created, what...
2
answers
1
votes
225
views
asked 5 months agolg...
I want to save my pyspark dataframe in RecordIO protobuf format. I am using Amazon EMR to run my pyspark scripts, and I want to use AWS SageMaker to train a machine learning model. SageMaker pipe mode...
1
answers
0
votes
172
views
asked 5 months agolg...
Hello,
I have a driver and executor pod template files exist in s3. But when I run the job, it failed wirh FileNotFoundException: s3:/<s3bucket>/podtemplate/driver.yaml (no such file or directory)...
1
answers
2
votes
220
views
asked 5 months agolg...
Hello,
Please share the difference between AWS Glue and AWS EMR and which one we should use and when?
Thanks,
1
answers
0
votes
1082
views
asked 5 months agolg...
In continuation to the ticket - https://repost.aws/questions/QUfxjbaGrXRTSKGy4-rnQ8Uw/how-to-upgrade-python-version-in-emr-since-python-3-7-support-discontinued , the latest version of EMR 6.14.0...
1
answers
0
votes
255
views
asked 5 months agolg...
I am facing issues using inline clustering and compaction in EMR, with the following error..
EMR : 6.13.0
Hudi: 0.13.1
com.esotericsoftware.kryo.KryoException: Unable to find class:...
0
answers
0
votes
91
views
asked 5 months agolg...
We have some parquet format files on AWS s3 and we want to create iceberg table with these files. Can Glue Crawler do this?
2
answers
0
votes
562
views
asked 5 months agolg...
I am using EMR 6.13.0(EMR on EC2) and it is present in eu-central-1 region. i have my s3 buckets in eu-central-1 and eu-west-1 region, i can be able to access the s3 buckets present in eu-central-1...
1
answers
1
votes
514
views
asked 5 months agolg...
I am trying to create a EMR cluster with version 6.13.0 and spark installed on it with version Spark 3.4.1 But the service role attached gives error that it does not have EC2 permissions.
I tried...
3
answers
0
votes
503
views
asked 5 months agolg...
EMR upgrade with AL2lg...
I have a emr 5.29 cluster running spark and hive application and received notification that AL1 will be unsupported after Dec 23. I am working on applications migration to emr 5.36. Questions,
1. What...
Accepted AnswerAmazon EMR
1
answers
1
votes
204
views
asked 6 months agolg...
I am running catalog queries like spark.catalog.tables, or show databases or show tables. All these commands failing emr_instance_profile role is not authorized to perform: glue:CreateDatabase on...
2
answers
3
votes
237
views
asked 6 months agolg...