- Newest
- Most votes
- Most comments
There are modules that cannot be added directly into the site-packages directory to be recognised inside an AWS Lambda environment. When that happens, you have to get an Amazon Linux image from Docker repositories and make your own compiled environment in a container version that will run on AWS Lambda
For example, if you want to use Python 3.6 a good choice will be amazonlinux:2018.03 in case you want to install more packages e.g. pandas, numpy, scipy
docker run -v $(pwd):/outputs -it amazonlinux:2018.03
Since Amazon Linux is based on Red Hat, you have to install via yum all dependencies when running docker and having already created your virtual environment
yum update -y
yum install -y \
python36 \
python36-devel \
python36-virtualenv \
python34-setuptools \
gcc \
gcc-c++ \
findutils \
rsync \
Cython \
findutils \
which \
gzip \
tar \
man-pages \
man \
wget \
make \
zip
For lxml you will also need
(lambda_docker) bash-4.2# yum install libxml2
...
(lambda_docker) bash-4.2# yum install libxslt
...
You install the module as usual
pip3.6 install lxml
You should see something like
(lambda_docker) bash-4.2# pip3.6 install lxml
Collecting lxml
Downloading https://files.pythonhosted.org/packages/2d/53/34a9f0c79c548e430148837892b6ae91adee571a0e8b6c17bd7ff9c2d12e/lxml-4.3.4-cp36-cp36m-manylinux1_x86_64.whl (5.7MB)
|################################| 5.7MB 2.0MB/s
Installing collected packages: lxml
Then, create your function lambda_function.py and add it in the zipped package pushd-ed and popd-ed from you docker session
from lxml import etree
def lambda_handler(event, context):
print(__name__)
print(etree.LXML_VERSION)
After created
zip -g site-packages.zip lambda_function.py
Before uploading, you can check that your zip file contains the lxml directories
\[jonathan@docker lambda_docker]$ unzip -l site-packages.zip
Archive: site-packages.zip
Length Date Time Name
--------- ---------- ----- ----
0 06-29-2019 23:09 __pycache__/
277 06-29-2019 23:09 __pycache__/easy_install.cpython-36.pyc
126 06-29-2019 23:09 easy_install.py
119 06-29-2019 23:29 lambda_function.py
0 06-29-2019 23:21 lib/
0 06-29-2019 23:39 lxml/
0 06-29-2019 23:37 lxml-4.3.4.dist-info/
4 06-29-2019 23:37 lxml-4.3.4.dist-info/INSTALLER
2954 06-29-2019 23:37 lxml-4.3.4.dist-info/METADATA
13384 06-29-2019 23:37 lxml-4.3.4.dist-info/RECORD
109 06-29-2019 23:37 lxml-4.3.4.dist-info/WHEEL
5 06-29-2019 23:37 lxml-4.3.4.dist-info/top_level.txt
7668 06-29-2019 23:37 lxml/ElementInclude.py
551 06-29-2019 23:37 lxml/__init__.py
0 06-29-2019 23:37 lxml/__pycache__/
3331 06-29-2019 23:37 lxml/__pycache__/ElementInclude.cpython-36.pyc
...
Now upload the zip and create an empty test {} in your lambda function
Result
START RequestId: bb240a17-c2dd-4d63-92c8-fa7561c09f64 Version: $LATEST
lambda_function
(4, 3, 4, 0)
END RequestId: bb240a17-c2dd-4d63-92c8-fa7561c09f64
REPORT RequestId: bb240a17-c2dd-4d63-92c8-fa7561c09f64 Duration: 0.30 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 50 MB
Relevant content
- asked 5 months ago
- Accepted Answerasked 2 years ago
- AWS OFFICIALUpdated 10 months ago
- AWS OFFICIALUpdated 5 months ago
- AWS OFFICIALUpdated 5 months ago
- AWS OFFICIALUpdated 2 years ago