Python Layers and Requests import module

0

Hello I am quite new on AWS world and facing some difficulties:

  1. I looked at lambda to transfer and manipulate (ETL) data sources into a database (through Stitch API). Most of my legacy data infrastructure is built like this (with Heroku Ruby, which I don't like)
  2. I have done most of the work fine using Python 3.9 AWS Lambda, but the last bit (post transfer to Stitch API) fails because that accepts only JSON data and the libraries I use (urllib, urlopen) only send byte encoded data (afaik)
  3. I read around and attempted to build a 3.8 Layer with the recommended python 'requests' module, I managed to build a 3.8 layer, I moved my lambda function back to 3.8.
  4. Now that's failing because "urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'OpenSSL 1.0.2k-fips 26" Reading around I should move to 3.10, but that is not available in amazon-linux-extras when creating a layer. I read that I should build it myself from sources, which seems a bit too far for me, opening other cans of worms.

Do you think the direction of work is fine? Do you have suggestion how to proceed, without getting into other difficulties? I thought that Python, AWS working with simple APIs would be quite straightforward, but it seems it is not. Thanks so much for your help Denis

질문됨 9달 전4160회 조회
2개 답변
1
수락된 답변

Hello.
There is a Lambda layer created by volunteers.
Using this one you may not have to create layers.
https://github.com/keithrozario/Klayers

profile picture
전문가
답변함 9달 전
profile picture
전문가
검토됨 9달 전
1

I got a DEFAULT_CIPHERS' from 'urllib3.util.ssl_' error when I trying a post request from a lambda function to an external API as well. I fix the problem installing a previous version of urllib3 ( The latest version has problems with boto3 ).

Try creating a new zip file to import requests, but this time after make the pip install requests, force again the instalation of urllib3==1.26.18, like this:

pip install --target=. --implementation cp --python-version 3.10 --only-binary=:all: urllib3==1.26.18 --upgrade

After that I got rid of the problem.

profile picture
David
답변함 5달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인