Add a layer to AWS lambda funtion

Question

I have a python notebook with pandas-profiling imported in order to generate reports for variables. Everything works well on local machine. Now I want to deploy it on AWS because eventually I want it working on large dataset in S3. But adding pandas-profiling as a layer for lambda function exceeding the maximum size (250 MB), I try to break pandas-profiling into several small folders and upload as layers but it doesn't work when importing it. Would anyone provide some suggestions? Or do anyone know the similar service on AWS as pandas-profiling? Thanks!

Answer

hi, for those cases where your Lambda function package size is greater than 250MB, you have the option to package and deploy your Lambda function as a docker container image, which supports up to 10GB, including all layers. Please see more in
[https://docs.aws.amazon.com/lambda/latest/dg/lambda-deploy-functions.html]() and
[https://docs.aws.amazon.com/lambda/latest/dg/images-create.html]()

Add a layer to AWS lambda funtion

Relevant content