No log groups or logs in Cloudwatch after sending data via Watchtower

0

I am trying to use the watchtower API to send logs to Cloudwatch - but I don't see any log groups or any data.

Here is the relevant python code. Am I instantiating the Cloudwatch handler correctly?

fwiw, under IncomingLogEvents: Sum, I do see that CloudWatch has gotten the logs! But I do not have any LogGroups at all.

Thanks.

class FileOps:
    def __init__(self):
        load_dotenv()
        self._service_name = "s3"
        self._region_name = os.getenv("AWS_REGION")
        self._aws_access_key_id = os.getenv("AWS_ACCESS_KEY_ID")
        self._secret_access_key = os.getenv("AWS_SECRET_ACCESS_KEY")
        self._endpoint_url = f"https://s3.{self._region_name}.amazonaws.com"
        self._session = self.get_thread_safe_s3_session()

        # Set up basic logging configuration
        logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
        self.logger = logging.getLogger(__name__)

        # Initialize CloudWatch handler
        self.initialize_cloudwatch_handler(self.logger)

    def get_cloudwatch_logs_client(self) -> Any:
        session = self.get_thread_safe_s3_session()
        client = session.client(
            "logs",
            region_name=self._region_name
        )
        return client

    def initialize_cloudwatch_handler(self, logger):
        # Set up CloudWatch handler
        cloudwatch_handler = watchtower.CloudWatchLogHandler(
            log_group=os.getenv("CLOUDWATCH_TIINGO_LOG_GROUP"),
            stream_name="TiingoSoraStocks-{strftime:%Y-%m-%d}",
            use_queues=True,  # Set to True for better performance in production
            boto3_client=self.get_cloudwatch_logs_client(),
            create_log_group=True
        )

        # Add CloudWatch handler to the logger
        logger.addHandler(cloudwatch_handler)
1 Answer
0

Your issue with no log groups or logs appearing in CloudWatch despite sending data could be due to several factors. Ensure the log group name in CLOUDWATCH_TIINGO_LOG_GROUP is correct and that create_log_group=True is functioning.

Verify that the log stream name format is correct and check permissions for creating log groups and streams.

Additionally, ensure your boto3 client is properly configured and consider adding error handling to catch any issues. Check the CloudWatch console later as there might be a delay in log visibility.

profile picture
EXPERT
answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions