Unable to compile aws-sdk-go-v2/service/ec2 programs in BUILD_GENERAL1_SMALL CodeBuild instances

Question

When attempting to compile Go programs that import github.com/aws/aws-sdk-go-v2/service/ec2  
  in AWS CodeBuild using BUILD_GENERAL1_SMALL compute type (3 GB RAM, 2 vCPU), the build fails 
  with the Go compiler being killed by the OOM killer:

go build -o bin/ics ./cmd/ics/
  github.com/aws/aws-sdk-go-v2/service/ec2:
  /root/.goenv/versions/1.24.8/pkg/tool/linux_amd64/compile: signal: killed
  make: *** [Makefile:13: build-ics] Error 1

The signal: killed indicates the Linux OOM killer terminated the process due to memory       
  exhaustion.

Environment

- CodeBuild Image: aws/codebuild/standard:7.0
  - Compute Type: BUILD_GENERAL1_SMALL (3 GB RAM, 2 vCPU)
  - Go Version: 1.24
  - SDK Version: github.com/aws/aws-sdk-go-v2/service/ec2 v1.261.0

What We Tried

1. Set GOMAXPROCS=2: Reduced Go compiler parallelism to limit concurrent compilation. Still  
  OOM.
  2. Set GOMAXPROCS=1: Forced completely sequential compilation. Still OOM.
  3. Enabled S3 caching: Cached /go/pkg/mod across builds so dependencies don't need to be     
  re-downloaded. Helped with build time but not memory usage during compilation.
  4. Pre-downloaded modules: Used go mod download in the install phase. This didn't help       
  because the OOM occurs during compilation, not download.

Observations

- The aws-sdk-go-v2/service/ec2 package is exceptionally large compared to other AWS service 
  packages
  - Other services like s3, dynamodb, cloudformation compile successfully on SMALL instances   
  - The issue is specific to the ec2 package compilation phase
  - Using BUILD_GENERAL1_MEDIUM (7 GB RAM) resolves the issue

Questions

1. Is there a way to reduce memory usage when compiling the aws-sdk-go-v2/service/ec2        
  package?
  2. Are there build flags or Go compiler options that could help reduce memory footprint      
  during compilation?
  3. Is this a known limitation of the EC2 service package due to its size?
  4. Are there plans to optimize the SDK's memory footprint during compilation?

Workaround

Currently using BUILD_GENERAL1_MEDIUM for projects that compile programs importing the EC2   
  service package, but this doubles our CodeBuild costs for these projects.

Tags

re:Post Agent · Answer

Based on your description, you're encountering an out-of-memory (OOM) issue when compiling Go programs that import the AWS EC2 SDK on CodeBuild's BUILD_GENERAL1_SMALL instances.

The issue you're experiencing is likely due to the limited disk space and memory available in the BUILD_GENERAL1_SMALL compute type. This compute type provides only 3 GB of memory, which appears insufficient for compiling the particularly large EC2 service package in the AWS SDK for Go.

To address your questions:

1. There isn't a straightforward way to significantly reduce memory usage when compiling the aws-sdk-go-v2/service/ec2 package, as the memory requirements are largely determined by the package's size and complexity.

2. While you've already tried reducing GOMAXPROCS, which was a good approach, the EC2 package is likely too large for the available memory regardless of parallelism settings.

3. The EC2 service package is indeed exceptionally large compared to other AWS service packages due to the comprehensive API surface area of EC2. This makes it more memory-intensive during compilation.

4. Your current workaround of using BUILD_GENERAL1_MEDIUM is the recommended solution. This compute type provides 7 GB of memory, which is sufficient for compiling the EC2 service package.

Unfortunately, there's no magic configuration that will allow the EC2 SDK to compile in the smaller environment. The recommended solution is to continue using the BUILD_GENERAL1_MEDIUM compute type for projects that require the EC2 service package. While this does increase costs, it's necessary to provide the resources required for compilation.

If cost is a significant concern, you might consider structuring your project to separate EC2-dependent code into a separate module or repository that's built less frequently, while keeping your main codebase on the smaller compute type.  
**Sources**  
[Troubleshooting AWS CodeBuild - AWS CodeBuild](https://docs.aws.amazon.com/codebuild/latest/userguide/troubleshooting.html)  
[Build environment compute modes and types - AWS CodeBuild](https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-compute-types.html)

Unable to compile aws-sdk-go-v2/service/ec2 programs in BUILD_GENERAL1_SMALL CodeBuild instances

Add your answer

Relevant content