Questions tagged with Amazon EMR Serverless
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I'm running into an issue where I have certain properties, such as EMR secret resolution and package resolution that are not working when I define them as spark defaults in the job configuration file...
0
answers
0
votes
126
views
asked 9 months agolg...
We have a pyspark job which we are executing to connect with MongoDB using the mongo-spark-connector.
The job is executed successfully with no errors in the stdout log file and in the stderr log file...
1
answers
0
votes
479
views
asked 9 months agolg...
Please help me understand the difference between the three( EMR , ECS And EMR studio) , and when any of these should be used as all three are used for managing and creating clusters. Thanks
1
answers
0
votes
590
views
asked 9 months agolg...
Getting this error when trying to run a simple spark job which reads a json file from s3 and prints the...
3
answers
0
votes
549
views
asked 9 months agolg...
I am using EMR 6.13.0, it is using python 3.7. in my code i have used boto3, the boto3 support for python 3.7 will be discontinued from December-2023.
and as we aware the python 3.7 support stopped...
2
answers
1
votes
4528
views
asked 9 months agolg...
I'm new to EMR serverless.
i found out that EMR Serverless uses client mode as default deploy mode.
and there's no informations to use cluster mode in emr serverless.
is there any way to use 'cluster...
1
answers
0
votes
376
views
asked 10 months agolg...
Hey Guys
I want to run my pyspark on EMR Serverless but it has some dependencies/libraries which are needed by the pyspark script to run. Please suggest a optimized approach to import the...
1
answers
0
votes
421
views
asked 10 months agolg...
Hello everyone!
I'm seeking advice on architecture design using AWS, specifically regarding the feature store process. Currently, I'm in the prototyping phase and using the tsfresh library for...
1
answers
0
votes
239
views
asked a year agolg...
I have a data of 225+ million in my Redshift Table. This data is the activity logs of the user who are coming and going at that time after scanning the door like the access logs of the user at what...
3
answers
0
votes
886
views
asked a year agolg...
I've been reading through documentation, but not able to find clear instruction on setup hive metastore in S3 for EMR Serverless, I only see examples of use glue cagtalog or aurora rds sql database....
1
answers
0
votes
946
views
asked a year agolg...
Json data is being considered as string while loading data from postgres to json file by AWS Gluelg...
I want to migrate postgres data to redshift, but I have a lot of jsonb data in postgres so for that I had given SUPER data type in Redshift but the problem here is while loading the data to redshift...
1
answers
0
votes
1049
views
asked a year agolg...
I'm having a lot of problems with disk space in emr serveless :
````
org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorter@1101b82b : No space left on device
````
I have set disk space...
Accepted AnswerAmazon EMR Serverless
2
answers
0
votes
1238
views
asked a year agolg...