By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Resourceinitializationerror: unable to pull secrets or registry auth: context deadline exceeded

0

Hi Team, I'm trying to use AWS Batch service with ECS Fargate. It's basically a python script to fetch the db password stored as secret from AWS SSM Parameter and run an ETL function.

I have ensured networking(internet access with NAT Gateway) and the required iam permission(Full Access) to fetch the secrets or ecr image. It is scheduled to run on an hourly basis. Sometimes, it is working fine but some other time it is failing with the below message.

"Resourceinitializationerror: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve secrets from ssm: service call has been retried 5 time(s): RequestCanceled: request context canceled caused by: context deadline exceeded"

This seems to be a strange issue. I'm happy to fix if any changes to be done from my side but i'm little worried on why it is unstable. Can some clarify on this issue please?

  • Henrique Santana
    a year ago

    Since it is an intermittent issue, it seems to be related to the subnet configuration (NACL?). Is it possible to share the networking configuration that you are using? e.g.: subnet conf, route table, SG, NACL, etc?

    All this would help to troubleshoot this. "context deadline exceeded" is usually a network issue. Also, if you have a Premium Support plan, I'd encourage you to open a case with them to get more help.

  • arunkumarwiz
    a year ago

    @Henrique Thanks for the response. We are using 2 private subnets for running AWS Batch fargate container. NACL is opened for all traffic ingress and egress. Route table is configured with Nat Gateway for Internet traffic. Egress of SG is opened to all traffic and ingress is empty. We do not have premium support. I have been running it as a cron schedule for every 2 hours. Success rate is around 70% and it is getting failed with the same error other times. Can you please let me know what else the issue maybe? Will this issue be fixed if i use AWS Secret manager instead of SSM Parameter store?

  • Henrique Santana
    a year ago

    "context deadline exceeded" errors are likely a network issue. We have this knowledge center article that might help you: https://aws.amazon.com/premiumsupport/knowledge-center/ecs-unable-to-pull-secrets/ Let me know if this helps you

Topics
ServerlessContainersManagement & GovernanceCompute
Tags
AWS FargateAWS Systems ManagerAWS BatchAmazon Elastic Container Service
Language
English
arunkumarwiz
asked a year ago260 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content