importing psycopg2 to glue job fails

0

Im using aws Glue in order to transfer from pg to s3. I wrote a script which seems to work.

Im trying to import psycopg2 to my script, And I keep getting the following error: "ModuleNotFoundError: No module named 'psycopg2'".

I tried adding the module both as a zip and whl to the s3 and then point the as a 'Python library path'.

It is important to note that my zip file is including the following folders:

  • psycopg2

  • psycopg2_binary.libs

  • psycopg2_binary-2.9.7.dist-info

I also tried adding it as a Job Parameter with all this options:

--additional-python-modules | psycopg2

--additional-python-modules | psycopg2==2.9.7

--additional-python-modules | psycopg2-binary

--additional-python-modules | psycopg2-binary==2.9.7

--extra-py-files | psycopg2

--extra-py-files | psycopg2==2.9.7

--extra-py-files | psycopg2-binary

--extra-py-files | psycopg2-binary==2.9.7

still nothing seems to work.

itamar
已提問 8 個月前檢視次數 374 次
1 個回答
0

When you say "using Glue", I guess you mean "Glue Shell" otherwise it wouldn't make sense to use a Python library for that. --additional-python-modules psycopg2-binary does work, check your logs, maybe you are running in a VPC and don't have Pypi access or you are using the old shell version.

profile pictureAWS
專家
已回答 8 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南