- Newest
- Most votes
- Most comments
The issue you're experiencing is that the describe_db_log_files() method only returns information about log files that are still accessible on the DB instance itself, not the logs that have been exported and stored in CloudWatch Logs. When Aurora publishes logs to CloudWatch Logs, they become separate from the instance's local log files.
To get the actual storage consumption of your Aurora logs in CloudWatch Logs, you'll need to use the CloudWatch Logs API instead of the RDS API. Specifically, you should use the describe_log_groups() method from the CloudWatch Logs client to get storage information.
Here's the approach:
- Use the CloudWatch Logs client (not RDS client)
- Look for log groups with the prefix
/aws/rds/cluster/followed by your cluster names - The
describe_log_groups()response includes astoredBytesfield that shows the total bytes stored for each log group
Here's a code example:
import boto3 cloudwatch_logs = boto3.client('logs') # Get all RDS-related log groups paginator = cloudwatch_logs.get_paginator('describe_log_groups') total_bytes = 0 for page in paginator.paginate(logGroupNamePrefix='/aws/rds/'): for log_group in page['logGroups']: total_bytes += log_group.get('storedBytes', 0) print(f"{log_group['logGroupName']}: {log_group.get('storedBytes', 0) / (1024**3):.2f} GB") print(f"\nTotal storage: {total_bytes / (1024**3):.2f} GB")
This will give you the actual CloudWatch Logs storage consumption across all your Aurora clusters, which should match the 100GB+ you're seeing in the console.
Sources
Publishing database logs to Amazon CloudWatch Logs - Amazon Aurora
Publishing Aurora PostgreSQL logs to Amazon CloudWatch Logs - Amazon Aurora
Publishing Amazon Aurora MySQL logs to Amazon CloudWatch Logs - Amazon Aurora
Monitoring log events in Amazon CloudWatch - Amazon Aurora
Relevant content
asked 5 months ago
