Whether you're taking your first steps with Kubernetes or you're an experienced practitioner looking to sharpen your skills, our Amazon EKS workshop series delivers practical, real-world experience that moves you forward. Learn directly from AWS solutions architects and EKS specialists through hands-on sessions designed to build your confidence with Kubernetes. Register now and start building with Amazon EKS!
How do I resolve "ImportError: No module named" in AWS Glue?
I want to resolve the "ImportError: No module named" error that I receive in AWS Glue.
Resolution
Missing third-party libraries
If you run an AWS Glue Python shell job that can't find the required Python module or package, then you receive an error message that's similar to the following:
"ImportError: No module named pyarrow.compat"
To resolve this issue, provide the missing library as an .egg or .whl file. For instructions, see Using an Egg or Whl file.
Note: AWS Glue Python shell jobs support .egg and .whl files. Python directly imports from either file type.
Important: Python shell jobs support Python 3. For more information, see AWS Glue version support policy.
Incorrect import statement in custom code
When you try to import custom code that’s packaged as an .egg or .whl file, you must use absolute imports. If you use relative imports, then you receive an "ImportError: No module named" error message.
To resolve this issue, modify your code’s import statement from relative imports to absolute imports.
Related information
How do I use external Python libraries in my AWS Glue ETL job?
For me, the issue was simple: when you package your code, generally we keep our main runner inside the package, hence the module calling is local, but when glue takes the package, the main runner is outside the package, so you need to do absolute import, not a relative. For example, package-->logger, extractor, loader main.py --> import package.logger, package.extractor, package.loader
Thank you for your comment. We'll review and update the Knowledge Center article as needed.
Relevant content
- asked 3 years ago
- AWS OFFICIALUpdated 10 months ago
