Python Layers and Requests import module

0

Hello I am quite new on AWS world and facing some difficulties:

  1. I looked at lambda to transfer and manipulate (ETL) data sources into a database (through Stitch API). Most of my legacy data infrastructure is built like this (with Heroku Ruby, which I don't like)
  2. I have done most of the work fine using Python 3.9 AWS Lambda, but the last bit (post transfer to Stitch API) fails because that accepts only JSON data and the libraries I use (urllib, urlopen) only send byte encoded data (afaik)
  3. I read around and attempted to build a 3.8 Layer with the recommended python 'requests' module, I managed to build a 3.8 layer, I moved my lambda function back to 3.8.
  4. Now that's failing because "urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'OpenSSL 1.0.2k-fips 26" Reading around I should move to 3.10, but that is not available in amazon-linux-extras when creating a layer. I read that I should build it myself from sources, which seems a bit too far for me, opening other cans of worms.

Do you think the direction of work is fine? Do you have suggestion how to proceed, without getting into other difficulties? I thought that Python, AWS working with simple APIs would be quite straightforward, but it seems it is not. Thanks so much for your help Denis

已提問 9 個月前檢視次數 4158 次
2 個答案
1
已接受的答案

Hello.
There is a Lambda layer created by volunteers.
Using this one you may not have to create layers.
https://github.com/keithrozario/Klayers

profile picture
專家
已回答 9 個月前
profile picture
專家
已審閱 9 個月前
1

I got a DEFAULT_CIPHERS' from 'urllib3.util.ssl_' error when I trying a post request from a lambda function to an external API as well. I fix the problem installing a previous version of urllib3 ( The latest version has problems with boto3 ).

Try creating a new zip file to import requests, but this time after make the pip install requests, force again the instalation of urllib3==1.26.18, like this:

pip install --target=. --implementation cp --python-version 3.10 --only-binary=:all: urllib3==1.26.18 --upgrade

After that I got rid of the problem.

profile picture
David
已回答 5 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南