How can I use python libraries spacy in aws glue jupyter notebook ?

Question

I tried using !pip3 install spacy to download the library but the notebook was not able to find the module, next I realized you can use %additioanl_python_modules, but I am not able to figure out how to do so...so I have to zip them even if I am not using my own custom libraries.

Answer

Have you tried using S3 to upload your library and the reference it in Glue?
https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#aws-glue-programming-python-libraries-zipping:~:text=.zip%20file.-,Loading,-Python%20libraries%20in

Answer

If you have a zip you need to use %extra_py_files and point to s3, if you use %additional_python_modules you can point to a whl on s3 or just use a pip package name list.  
Notice you need to do this before you run any other cells that would start the session.

How can I use python libraries spacy in aws glue jupyter notebook ?

Contenuto pertinente