Codebuild suddenly fails in DOWNLOAD_SOURCE phase

Question

Hello,  
Starting today our codebuild project, which has been working flawlessly for months and not modified has stopped working.  
Failure occurs in DOWNLOAD_SOURCE phase, with this message: "SINGLE_BUILD_CONTAINER_DEAD: Build container found dead before completing the build. Build container died because it was out of memory, or the Docker image is not supported."  
The applications code is hosted on CodeCommit.  
We did not do any modification to the buildspec file, and the very same build was even still working yesterday.  
  
We are using aws/codebuild/amazonlinux2-x86_64-standard:2.0 images, and did not do any modifications but the failure suddenly happened today. During the previous build the job was using at most 20% of the available memory (15GB)  
Sadly nothing gets logged because it fails even before processing the first commands of the buildspec so it's nearly impossible to debug.   
  
Do you guys have any ideas on what could cause this ?  
  
Thanks,  
Daniel  
  
Edited by: drolland on Oct 20, 2020 2:29 AM  
  
Edited by: drolland on Oct 20, 2020 2:30 AM

drolland · Answer

So I finally found the reason by creating some test build, so I'm sharing the solution I found below. **Solution** If you are using multiple sources for your build project, **get sure that Git Clone Depth is set to 1 for all your sources**.
Git clone depth defaults to "Full", so if you have built your project with CloudFormation get sure to include _GitCloneDepth: 1_ in your template SecondarySources list. **Explanation** Not sure about what has changed internally, but there seems to be some kind of timeout when cloning the repos, even though the error message is unclear about that.
As of today (2020-10-20), this error is reproductible if you have multiple repos containing enough commits, by creating an empty test project and including 3 additional sources. The build will fail with the same error message before starting. Once I was able to reproduce the issue, I've tried fiddling around with clone depth and it finally worked. Again, I'm not sure if this is temporary or will be fixed, but as of yesterday (2020-10-19), this workaround was not necessary. Edited by: drolland on Oct 20, 2020 5:37 AM

AWS-User-6139167 · Answer

Hi drolland,  
  
Would you PM me build ARNs of affected builds? Preferably one before and one after you noticed this behavior change. I'd like to help investigate this further.

Codebuild suddenly fails in DOWNLOAD_SOURCE phase

Relevant content