Questions tagged with Amazon EMR Serverless
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
We are comparing the emr/eks and emr serverless offer. We are trying to achieve the tpc-ds perf benchmark through (https://github.com/aws-samples/emr-on-eks-benchmark). We are not able to do it on...
1
answers
0
votes
897
views
asked 2 years agolg...
I am processing a dataset and need to submit a job to EMR serverless for the dataset to be processed in a distributed way. I have created an application in EMR studio. I would like to submit jobs to...
2
answers
0
votes
1711
views
asked 2 years agolg...
I have created a Spark job with Scala and I am trying to find the way to get the logs into cloudwatch.
So far I have tried to package the job as an uber Jar with a [cloudwatch...
Accepted AnswerAmazon EMR Serverless
1
answers
0
votes
1400
views
asked 2 years agolg...
Hi,
We are planning to make use of EMR Serverless Hive service to move one of our work load from EMR persistent cluster. This particular workload uses the external XML serde to define hive table. In...
1
answers
0
votes
814
views
asked 2 years agolg...
EMR on EC2 provides job flow metadata via `/mnt/var/lib/info/job-flow.json` file on disk. However, this doesn't exist on EMR on EKS. Is there an equivalent metadata provider to get Id of the job run?
1
answers
0
votes
558
views
asked 2 years agolg...
Currently for EMR on EC2 we pass in job configuration using a local file `my.jar --cfg /local/path/config.cfg ....`. Now I'm migrating jobs to both EMR on EKS and EMR Serverless. How can I achieve the...
1
answers
0
votes
502
views
asked 2 years agolg...
I know Step Functions use AWS SDK and it gets updated regularly so my questions:
1. What is the SDK update cycle for Step Functions?
2. Do we know if EMR Serverless can be integrated without using...
1
answers
1
votes
532
views
asked 2 years agolg...
Hello,
How do we pass application.conf or properties file to spark jobs on emr serverless, in standard EMR we copy properties or conf files to master node /home/hadoop/?
1
answers
0
votes
694
views
asked 2 years agolg...
I am using both pyspark and local python kernel (%%local) in a single EMR notebook. I am able to install packages successfully in pyspark kernel using EMR bootstrap but I am unable to install...
2
answers
1
votes
3765
views
asked 2 years agolg...
I am trying to invoke a spark script that runs over EMR serverless from a Step function. Nevertheless, I have not found an out of the box integration. One option is to use a lambda function that...
1
answers
1
votes
1997
views
asked 2 years agolg...
Is it possible to run a JDK11 application on EMR serverless? I tried and got an error:
```
Job execution failed, please check complete logs in configured logging destination. ExitCode: 1. Last few...
Accepted AnswerAmazon EMR Serverless
4
answers
0
votes
2379
views
asked 2 years agolg...
I'm trying to create an isolated Python virtual environment to package Python libraries necessary for a Pyspark job.
I was successful to make it work by simply following these steps...
2
answers
0
votes
989
views
asked 2 years agolg...