How do you manage Batch Job Descriptions when the ECR container image changes?
I have several AWS Batch Job Definitions that rely on container images stored in ECR. My problem is that whenever I update the container images in ECR I have to update the Job Definitions as well.
My first instinct was to use image tags (development, testing, release, etc.) and hard code my Job Definition to point to the correct tag. However, all the "best practices" documents say that your ECR repos should be immutable, which makes it impossible to re-tag an image as it moves through the application lifecycle. (development-->testing, testing-->release).
Another option (which is what I'm using) is to update Job Definitions every time the ECR image gets rebuilt. This works but it feels like a game of cat-and-mouse where I have to make sure that I find and update everywhere that image is used.
I've only started with ECS / Batch but already it feels like trying to keep everything updated and pointed in the right direction is a losing proposition. Am I missing something? Is there an easier way? How are you managing your Batch Job Descriptions when it comes to ECR image update?
UPDATE: this seems to be a related question but it doesn't really provide an answer - https://repost.aws/questions/QUrBwePY4tSeut4XFlQsyhAQ/recommended-batch-automated-workflow-for-updating-docker-containers
- Newest
- Most votes
- Most comments
Hello.
Another option (which is what I'm using) is to update Job Definitions every time the ECR image gets rebuilt. This works but it feels like a game of cat-and-mouse where I have to make sure that I find and update everywhere that image is used.
I often update using this method.
It is also possible to automate this by introducing CI/CD and updating the task definition when the container image is pushed to ECR.
CodePipeline can launch CI/CD using ECR as a source.
https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-ECR.html
Relevant content
- asked 4 months ago
- asked 2 years ago
AWS OFFICIALUpdated 3 years ago- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 19 days ago
We are using GitHub Actions for our CI/CD and I'm using "aws batch describe-job-definitions" and "aws batch register-job-definition" CLI commands to do most of the heavy lifting. It works but it's a bit clumsy and requires us to keep track of every single use of that container image in AWS and update it every time a new image it pushed to ECR. There seems like there should be a better way.