AgentCore Runtime: Container starts but handler never invoked, no CloudWatch logs

When working with Amazon Bedrock AgentCore Runtime, the issue you're experiencing with no handler invocation and empty CloudWatch logs is common and can be addressed through several troubleshooting steps.

First, regarding the invocation protocol, Amazon Bedrock AgentCore Runtime expects your container to expose an HTTP server on port 8080 with the /invocations path available. Your current Dockerfile is using the Lambda Runtime Interface Client (awslambdaric), which is not the correct approach for AgentCore Runtime.

Here's what you should check and modify:

1. Container Configuration: Your container needs to run a web server that listens on port 8080 and handles POST requests to the /invocations path.

2. CloudWatch Logs: Ensure you're looking at the correct log group. The standard pattern is:

/aws/bedrock-agentcore/runtimes/<agent_id>-<endpoint_name>/runtime-logs

Double-check that you're looking at the right log group for your specific agent ID and endpoint name.

3. IAM Permissions: Verify that your execution role has the necessary permissions for CloudWatch Logs. The role needs permissions for:

• logs:DescribeLogStreams
• logs:CreateLogGroup
• logs:DescribeLogGroups
• logs:CreateLogStream
• logs:PutLogEvents

4. Dockerfile Example: Here's a more appropriate approach for AgentCore Runtime:

FROM --platform=linux/arm64 python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
ENV PYTHONUNBUFFERED=1
EXPOSE 8080
CMD ["python", "app.py"]

5. Application Code: Your app.py should implement a web server (using Flask, FastAPI, etc.) that listens on port 8080 and handles POST requests to /invocations.

6. Diagnostic Steps:

• Try running your container locally with the same payload to see if it works
• Add verbose logging throughout your code
• Check if you need to set ECS_ENABLE_AWSLOGS_EXECUTIONROLE_OVERRIDE for proper logging

7. OTEL Configuration: If you're using observability features, ensure your OTEL environment variables are correctly set:

AGENT_OBSERVABILITY_ENABLED=true
OTEL_PYTHON_DISTRO=aws_distro
OTEL_PYTHON_CONFIGURATOR=aws_configurator
OTEL_RESOURCE_ATTRIBUTES=service.name=<agent-name>,aws.log.group.names=/aws/bedrock-agentcore/runtimes/<agent-id>,cloud.resource_id=<AgentEndpointArn:AgentEndpointName>

By implementing these changes, you should be able to get your agent working properly with AgentCore Runtime and see logs appearing in CloudWatch.
Sources
Troubleshoot AgentCore Runtime - Amazon Bedrock AgentCore
IAM Permissions for AgentCore Runtime - Amazon Bedrock AgentCore
Troubleshoot stopped Windows tasks in Amazon ECS | AWS re:Post
Add observability to your Amazon Bedrock AgentCore resources - Amazon Bedrock AgentCore

I've deployed 12 versions of an agent to Amazon Bedrock AgentCore Runtime in ap-southeast-1, trying different configurations based on AWS documentation. The container starts successfully (Status: Ready), endpoint is Active, but the handler is never invoked and no logs appear in CloudWatch.

Environment

• Region: ap-southeast-1
• Agent: hosted_agent_dwqon
• Runtime ID: hosted_agent_qvade-4dddddd
• Container: ARM64, Python 3.11, 191 MB
• Agent Status: Ready ✅
• Endpoint Status: Active ✅
• Log Groups: Both exist but empty (0 bytes) ❌
  • /aws/bedrock-agentcore/runtimes/hosted_agent_qvade-4dddddd-DEFAULT
  • /aws/vendedlogs/bedrock-agentcore/runtime/APPLICATION_LOGS/hosted_agent_dwqon-3XUjLe6b8f

12 Versions Deployed - Summary

Current Configuration (Version 12)

Dockerfile

FROM --platform=linux/arm64 python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --upgrade pip && \
    pip install --no-cache-dir -r requirements.txt
COPY . .
ENV PYTHONPATH=/app
ENV PYTHONUNBUFFERED=1
ENV AGENT_OBSERVABILITY_ENABLED=true
ENV OTEL_PYTHON_DISTRO=aws_distro
ENV OTEL_PYTHON_CONFIGURATOR=aws_configurator
EXPOSE 8080
CMD ["gunicorn", "--bind", "0.0.0.0:8080", "--workers", "1", "--timeout", "300", "--access-logfile", "-", "--error-logfile", "-", "--log-level", "info", "app_server:app"]
app_server.py (Flask Web Server)
import logging
from flask import Flask, request, jsonify
from agent import handler, initialize_agent

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

app = Flask(__name__)

logger.info("Starting AgentCore Runtime web server...")
initialize_agent()
logger.info("Agent initialized successfully")

@app.route('/health', methods=['GET'])
def health_check():
    logger.info("Health check requested")
    return jsonify({"status": "healthy"}), 200

@app.route('/invocations', methods=['POST'])
def invoke_handler():
    logger.info("Handler invocation requested")
    event = request.get_json()
    result = handler(event, None)
    logger.info("Handler completed successfully")
    return jsonify(result), 200

If your issue is about not seeing the logs, ensure you have enabled tracing on your agent (check the console) and that the IAM roles associated with the agent has the right permissions (https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/runtime-permissions.html)

Alternatively use the bedrock starter toolkit to create and run your agent.