How to troubleshoot "layer conversion failed: Could not open certain files or directories"

Question

I am trying to set up a lambda function in python 3.12 via a .zipfile archive; it requires pyodbc, which is being provided (with the mssql drivers) by a third-party layer. I have deployed the code and it runs, but I am not able to add the layer, or any layer (even the stock AWS ones) to the function. Regardless of which layer or layer type is chosen, it always displays the error "Layer conversion failed: Could not open certain files or directories;". No further details appear to be available anywhere.

I have examined the zip file for the lambda function and it all appears to be correct; the code runs, to the point where it fails because pyodbc is missing, because the layer providing it can't be added to the function without triggering the above error.

There are no directory overlaps between the layer (which contain the directories "microsoft", "python", "lib" and "bin") and the lambda function zipfile. Since no additional diagnostic information is shown and it never reaches the logs, I'm not sure how to troubleshoot further.

Any advice? The build environment is Windows 11, the AWS runtime is x86_84.

Answer

It sounds like you have already followed the troubleshooting steps from the Lambda dev guide for [Layer conversion failed](https://docs.aws.amazon.com/lambda/latest/dg/troubleshooting-deployment.html#troubleshooting-deployment-LayerConversionFailed) [1].

As a follow on, ensure that the layer is packaged in required [format](https://docs.aws.amazon.com/lambda/latest/dg/python-layers.html#python-layer-paths) [2]
```
layer_content.zip
└ python
    └ lib
        └ python3.12
            └ site-packages
                └ ....
```
Check that the layer_content zip file has the python folder at the root of the zip file (it's easy to accidentally zip up the folder the python folder is in and end up with it one level down). You want to ensure you are just adding the folders needed to the zip and not zipping up the whole directory they are in, e.g `zip -r layer_content.zip python`

----

[1] https://docs.aws.amazon.com/lambda/latest/dg/troubleshooting-deployment.html#troubleshooting-deployment-LayerConversionFailed

[2]  https://docs.aws.amazon.com/lambda/latest/dg/python-layers.html#python-layer-path

Answer

So it turns out, after an exhaustive one-file-at-time test process, that there is a directory depth limit of 4 for files in the lambda function directory. It doesn't seem to be an issue by itself, but if there is an active layer, then having any file in a directory more than 4 levels down triggers the "Could not open certain files or directories" error, whether it's when trying to add a layer to a function that's been uploaded, or trying to upload a function when a layer is active.

I couldn't find any reference to this limit, but I had 2 files below 4 levels, and moving them up in the hierarchy fixed, or worked around, the issue.

Keeping everything flat is not an option; greenlet, for instance, puts greenlet.h 5 or 6 levels down in the \include directory, and this is just Python being Python. Module structures require directories, sometimes more than 4 levels deep.

Sharing this in case it helps anyone else. Thanks, @Gillian, for your efforts.

How to troubleshoot "layer conversion failed: Could not open certain files or directories"

Relevant content