EMR Jupyter Notebook: PySpark Imports Work in Shell, Not in Notebook- Issue is importing custom files

0

Issue: PySpark works in the first cells (likely SparkSession creation) but throws import errors when using my Python files in later cells.

Environment: AWS EMR ( Amazon EMR version emr-6.4.0 Installed applications JupyterEnterpriseGateway 2.1.0, JupyterHub 1.4.1, Spark 3.1.2)

Question: Why the import issue with custom files? Best practices for importing Python files in EMR notebooks?

Enter image description here

I tried all the possible ways of exporting like %run .. %run -i

Harish
asked 19 days ago220 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions