1 Risposta
- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
4
Hello,
Yes, it looks like either the permission issue or the tmp files might be in use.
- Please check if you try opening the pyspark shell as hadoop user in EMR or use sudo pyspark in hadoop user
- Try checking the
spark-shell
working without issue instead of pyspark shell. - Include
spark.local.dir
to different local directory in primary node to see if this fix the issue. - Restart the spark service(
sudo systemctl restart spark-history-server.service
- Set the log level to debug
rootLogger.level = debug
in log4j file/etc/spark/conf/log4j2.properties
and retry the pyspark shell. This might give more insight to understand the issue.
Contenuto pertinente
- AWS UFFICIALEAggiornata un anno fa
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 2 anni fa