How can I use python libraries spacy in aws glue jupyter notebook ?

Have you tried using S3 to upload your library and the reference it in Glue? https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#aws-glue-programming-python-libraries-zipping:~:text=.zip%20file.-,Loading,-Python%20libraries%20in

If you have a zip you need to use %extra_py_files and point to s3, if you use %additional_python_modules you can point to a whl on s3 or just use a pip package name list.
Notice you need to do this before you run any other cells that would start the session.