Intermittent Downtime with API and Frontend Access: EC2RoleProvider Error in Server Logs

Question

How can I address an issue where I'm experiencing intermittent downtime with both the APIs and the frontend of my application? Occasionally, the application operates smoothly, but at other times, neither the APIs nor the frontend are accessible. Upon reviewing the server logs, I encountered the following error: "ERROR EC2RoleProvider Failed to connect to Systems Manager with SSM role credentials. Error calling RequestManagedInstanceRoleToken: AccessDeniedException: Systems Manager's instance management role is not configured for account:...." What steps should I take to resolve this connectivity problem and ensure stable access to my application's components?

Answer

"AccessDeniedException" exception with respect to SSM service generally relates to SSM agent on able to fully perform its tasks using the Ec2 instance profile role. This can originate from multiple causes:
- Firstly check during the issue, is SSMAgent is up and running.
- If the EC2 instance profile role has 'AmazonSSMMManagedInstanceCore' policy attached to it.
- Check if any timeout messages are showing up in SSMAgent logs.
- You can also confirm if the instance is actually getting managed by SSM or not using this command: 
$ aws ssm describe-instance-associations-status --instance-id 
- Try to restart the SSM agent as well:
$ sudo systemctl stop amazon-ssm-agent
$ sudo systemctl disable amazon-ssm-agent

As a last measure, if there are no tasks running on the instance, try the below steps:
1. Detach the IAM role associated to the instance.
2. Wait-for a minute and re-attach the IAM role back to the instance.
3. Restart the SSM agent on both the instances and check if they are showing as managed instance in SSM Fleet Manager.

If nothing helps, please contact AWS Support for Systems Manager service.

Intermittent Downtime with API and Frontend Access: EC2RoleProvider Error in Server Logs

Relevant content