Updating ingested log events in CloudWatch and querying them with Logs Insights

Regarding your questions about updating ingested log events in CloudWatch and using CloudWatch Logs Insights:

1. Updating ingested log events: Unfortunately, it is not possible to update or modify a log event that has already been ingested into CloudWatch Logs. Once a log event is sent to CloudWatch, it becomes immutable. The CloudWatch Logs service is designed for append-only operations, meaning you can add new log events but cannot modify existing ones.

Given your use case of tracking API calls, a better approach would be to ingest separate log events for different stages of the API call. You could:

• Ingest a log event at the start of the API call
• Ingest another log event when the API call completes successfully
• Ingest an error log event if something goes wrong

This way, you maintain a complete record of the API call lifecycle without needing to update existing log events.

2. Using CloudWatch Logs Insights: Yes, you can use all the features of CloudWatch Logs Insights for your custom ingested logs. CloudWatch Logs Insights is designed to work with both AWS service logs and custom application logs. It provides a powerful query language and various commands to search, filter, and analyze your log data effectively.

Some key features you can use with your ingested logs include:

• Running interactive queries on your log data
• Using various query commands like filter, stats, sort, limit, etc.
• Visualizing query results in graphs
• Saving queries for future use
• Using natural language to generate and update queries

CloudWatch Logs Insights automatically discovers fields in JSON-formatted logs, which can make querying easier if you structure your log events accordingly.

By leveraging these capabilities, you can gain deep insights into your application's behavior, troubleshoot issues, and monitor the performance of your API calls effectively.

Sources
Analyzing log data with CloudWatch Logs Insights - Amazon CloudWatch Logs
CloudWatchLogsAsyncClient (AWS SDK for Java - 2.21.29)
Use natural language to generate and update CloudWatch Logs Insights queries - Amazon CloudWatch Logs