ECR login token expiry - reauthentication suggestions

0

We are seeing random ECR docker login token expiry errors in our Jenkins pipelines.

docker: Error response from daemon: pull access denied for my-account.dkr.ecr.us-east-1.amazonaws.com/my-project/myapp/myimage:myversion, repository does not exist or may require 'docker login': denied: Your authorization token has expired. Reauthenticate and try again.

As per AWS document, https://docs.aws.amazon.com/AmazonECR/latest/public/public-troubleshooting.html, we need to reauthenticate the ecr login. We use ecr get-login within our Jenkinsfile to run activities within a docker container as below,

sh '$(aws ecr get-login --no-include-email --region us-east-1)'
            withDockerContainer(image: 'myaccount.dkr.ecr.us-east-1.amazonaws.com/my-project/myapp/my-image:my-version'){

We were planning to update the ecr login section over a try-catch and ensure the reauthentication is done in the catch block. We would like to get a confirmation if there are any other better ways of handling this error.  

try {
                        sh '$(aws ecr get-login --no-include-email --region us-east-1)'
                    } catch (Exception e) {
                        sh 'docker logout "myaccount.dkr.ecr.us-east-1.amazonaws.com"'
                        sh '$(aws ecr get-login --no-include-email --region us-east-1)'
                    }
已提問 1 年前檢視次數 974 次
1 個回答
1
已接受的答案

If you set up a Jenkinsfile like this, don't you log in every time?
https://docs.aws.amazon.com/AmazonECR/latest/userguide/Registries.html
https://www.jenkins.io/doc/book/pipeline/docker/

pipeline{
    agent any
    stages{
        stage('Hoge') {
            steps{
                script{
                    docker.withRegistry("https://${AWSAccountID}.dkr.ecr.${ECR Region}.amazonaws.com") {
                        sh("`aws ecr get-login --no-include-email --region ${ECR Region}`")
                        docker.image('my-custom-image').inside {
                          sh("echo 'hoge'")
                        }
                    }
                }
            }
        }
    }
}
profile picture
專家
已回答 1 年前
profile pictureAWS
專家
已審閱 1 年前
  • AFAIK, this one is better than the try / catch: I don't see why the second login in cache block would succeed if initial on just failed. They will be quasi-simultanous. The issue that you described initially is more of a timeout happening long after initial login so that will be ineffective.

  • The token expiry happens quite randomly. It does not happen always. So if indeed the token has expired, we need to be doing reauthentication as per AWS suggestion. Hence, believed that the try catch will ensure that it will perform the reauthentication in case of failure.

  • Are you suggesting that changing to docker.withRegistry will solve the issue for us? May I know why the withDockerContainer gives back such issues?

  • I'm not sure why withDockerContainer would cause the token expiration issue, but I believe docker.withRegistry logs you in every time you run with a build.

  • Understood. I will give this a try. There is also a recommendation to cleanup current user docker credentials using sh 'rm ~/.docker/config.json || true' on https://docs.cloudbees.com/docs/cloudbees-ci-kb/latest/client-and-managed-controllers/withdockerregistry-step-fails-with-amazon-ecr and https://plugins.jenkins.io/amazon-ecr/. If I include that it is introducing delays in the docker image download and times out. Is the docker credentials cleanup necessary?

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南