Questions tagged with Amazon EMR
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I want my EMR cluster to be terminated automatically post an idle time.
I have configured 'Automatically terminate cluster after idle time' and set the idle time as '5 minutes' .
In my cluster i have...
1
answers
0
votes
238
views
asked 6 months agolg...
If my environment is full of Apache Hudi integrating with EMR and Lake Formation, I found out that Hudi environment is not very friendly to be used by Redshift nor Athena. There are many advanced...
1
answers
0
votes
451
views
asked 6 months agolg...
My customer is using AWS EMR and is storing all the Hive meta data on an external RDS instance, using MySQL 5.7.* And since MySQL 5.7 is running out of its lifecycle, we are pushing them to upgrade...
1
answers
1
votes
279
views
asked 6 months agolg...
Everyday a new emr cluster span up and terminated after completing the step job. Checking the cloudtrail, seems a Data Pipeline created it. I am not sure how to get more details like who created, what...
2
answers
1
votes
258
views
asked 6 months agolg...
I want to save my pyspark dataframe in RecordIO protobuf format. I am using Amazon EMR to run my pyspark scripts, and I want to use AWS SageMaker to train a machine learning model. SageMaker pipe mode...
1
answers
0
votes
199
views
asked 6 months agolg...
Hello,
I have a driver and executor pod template files exist in s3. But when I run the job, it failed wirh FileNotFoundException: s3:/<s3bucket>/podtemplate/driver.yaml (no such file or directory)...
1
answers
2
votes
247
views
asked 6 months agolg...
Hello,
Please share the difference between AWS Glue and AWS EMR and which one we should use and when?
Thanks,
1
answers
0
votes
1347
views
asked 6 months agolg...
In continuation to the ticket - https://repost.aws/questions/QUfxjbaGrXRTSKGy4-rnQ8Uw/how-to-upgrade-python-version-in-emr-since-python-3-7-support-discontinued , the latest version of EMR 6.14.0...
1
answers
0
votes
295
views
asked 6 months agolg...
I am facing issues using inline clustering and compaction in EMR, with the following error..
EMR : 6.13.0
Hudi: 0.13.1
com.esotericsoftware.kryo.KryoException: Unable to find class:...
0
answers
0
votes
102
views
asked 6 months agolg...
We have some parquet format files on AWS s3 and we want to create iceberg table with these files. Can Glue Crawler do this?
2
answers
0
votes
647
views
asked 6 months agolg...
I am using EMR 6.13.0(EMR on EC2) and it is present in eu-central-1 region. i have my s3 buckets in eu-central-1 and eu-west-1 region, i can be able to access the s3 buckets present in eu-central-1...
1
answers
1
votes
596
views
asked 6 months agolg...
I am trying to create a EMR cluster with version 6.13.0 and spark installed on it with version Spark 3.4.1 But the service role attached gives error that it does not have EC2 permissions.
I tried...
3
answers
0
votes
566
views
asked 6 months agolg...