- Newest
- Most votes
- Most comments
You are correct in your observation. EMR Serverless does not automatically refresh or check for updates to images stored in Amazon ECR, even if you're using a single tag for your images.
When you create an EMR Serverless application with a custom image, it uses the image specified at the time of creation or last update. The application will continue to use this same image for subsequent job runs, regardless of whether the image in ECR has been updated.
To use a new version of your custom image, you need to explicitly update your EMR Serverless application. This involves specifying the new image URI when updating the application configuration.
Your approach of using a single tag for your images is not ideal for version control and reproducibility. As you've mentioned, it's a good practice to use immutable tags or image digests. This ensures that each version of your application is tied to a specific, unchanging image version.
When you're ready to update your application to use a new image version, you'll need to:
- Push the new image version to ECR with a new tag or digest.
- Update your EMR Serverless application configuration with the new image URI.
This manual update process gives you more control over when your application starts using a new image version, allowing for proper testing and validation before deployment.
Remember to ensure that your custom images are built on the correct base image for your EMR version, don't modify critical environment variables, and follow other best practices for EMR Serverless custom images.
Sources
ImageConfigurationInput - Amazon EMR Serverless
Customizing an EMR Serverless image - Amazon EMR
Relevant content
- asked 3 years ago
- asked 10 months ago
- asked 8 months ago
- AWS OFFICIALUpdated 2 years ago
