Skip to content

CloudWatch Logs Insights: filtering on a computed value (alias or coalesce) silently drops rows

0

Edit: the first version of this question said only an alias named user was affected. More testing showed that was wrong. A filter on any computed value can drop rows: an alias of any name, or coalesce() used directly inside filter. The details below replace the original post.

Summary

In CloudWatch Logs Insights (eu-central-1), a filter on a computed value returns fewer rows than the same filter written on raw fields. The query completes with status Complete and no error, so results are silently incomplete. Filters on raw fields, and @message like, returned the correct rows in every test.

Repro

Three log groups with ECS task logs (awslogs driver), one JSON object per line. The user id is logged as user_id by some services and userId by others:

{"level":"info","time":"2026-09-15T11:40:27.569Z","msg":"socket disconnected","user_id":"<24-hex id>","socket_id":"..."}
{"level":"info","time":"2026-09-15T11:40:27.572Z","msg":"Stored reconnection state for unintentional disconnect","userId":"<24-hex id>"}

Time range: the last hour. All queries were run against the same fixed start and end time and end with | stats count(*) as n.

FilterRowsCorrect
filter user_id = "<id>" or userId = "<id>"99
filter @message like "<id>"99
fields coalesce(user_id, userId) as uid | filter uid = "<id>"59
filter coalesce(user_id, userId) = "<id>" (no alias)59
fields coalesce(user_id, userId) as uid | filter uid like "<id>"59
filter user_id = "<id>"77
fields user_id as uid2 | filter uid2 = "<id>"57
filter strcontains(user_id, "<id>")77
  • The values are correct. When the 9 rows from the raw-field filter are displayed with fields coalesce(user_id, userId) as uid, uid equals <id> on all 9. Only filtering on it loses rows.
  • Which rows went missing: the 4 newest events, 22 to 58 seconds old when the query ran. They were still missing when the same fixed window was queried again 45 seconds later.
  • They appeared later. Querying the same fixed window again about 7.5 minutes later, when those events were 7 to 8 minutes old, every computed-value filter returned the correct count (9, 9 and 7). The raw-field filter returned them from the first run.
  • Functions on raw fields are fine. strcontains(user_id, ...) returned the correct rows. What breaks is filtering on a value created by fields ... as or by coalesce().

Older data

The same thing happens on older data, and results for the same fixed window change over hours but are stable between back-to-back runs.

  • 24-hour window, 2026-09-14 09:40 to 2026-09-15 09:40 UTC, correct answer 185 rows (raw-field filter).
    • fields coalesce(user_id, userId) as user | filter user = "<id>" returned 166 in the morning and 175 about two hours later, when three runs in a row all returned 175.
    • All 19 rows missing in the morning fell in one minute, 2026-09-15 09:27:56 to 09:28:59.
    • At the later check, the same query aliased uid, and the inline coalesce() filter, both returned 185.
  • 3-minute windows around the ten busiest minutes of a day.
    • fields coalesce(user_id, userId) as user | filter ispresent(user) returned 0 in 9 of 10 windows. ispresent(uid) returned 10 to 38 rows in the same windows.
    • Across five log groups, filter user = "<id>" was correct in 2 of the 3 windows tested and still wrong in the third (9 instead of 19).

Which slices fail therefore depends on the filter type, the alias, the set of log groups and when the query runs. That is why an earlier version of this question wrongly blamed the alias name user.

Ruled out

  • A real field with the alias name: in the windows checked, no log event has a user key at any nesting level (JSON key paths of every raw @message). The inline coalesce() filter has no alias at all and still drops rows.
  • Field indexes: there are no field index policies on the log groups or at account level.
  • Late ingestion: lag between @timestamp and @ingestionTime is normal (1 to 5 s).
  • Skipped events: recordsScanned is identical for the raw-field and computed-value queries, and estimatedRecordsSkipped is 0.
  • Account-specific: it reproduces in a second account in the same region, which has similar but higher-volume logs. On a 10-minute window queried as soon as it ended, the raw-field filter returned 75 rows for one id and both computed-value filters returned 55. All 20 missing events were 36 to 74 seconds old. The same fixed window queried 8 minutes later returned 75 for every filter. Earlier checks in that account looked clean only because their windows ended 15 minutes or more before the query.

Questions

  1. Are filters on computed values (fields ... as, coalesce()) evaluated differently from filters on raw fields, for example on recent data or on particular storage segments?
  2. Is this a known issue? Should filters on computed values give exact results?

Workaround

Filter on raw fields first, and compute aliases only for display:

filter user_id = "<id>" or userId = "<id>"
| fields @timestamp, coalesce(user_id, userId) as uid, msg

asked a month ago98 views

1 Answer
0

Based on the behavior you've described, it appears you've encountered an edge case in CloudWatch Logs Insights query processing. While user is not documented as a reserved keyword in the CloudWatch Logs Insights query language, your systematic testing strongly suggests it's being treated specially by the query engine in certain contexts.

To answer your questions:

  1. Is user treated specially? While not officially documented, your evidence indicates that user may indeed be handled differently during query optimization or execution, particularly when used as an alias derived from log fields and combined with certain filter operations. The fact that the issue is deterministic, affects specific time slices, and disappears with alternative filter syntax (like like or strcontains) or when using a constant value suggests internal query processing logic is treating this identifier specially.

  2. Is this a known issue? There is no publicly documented list of identifiers that should be avoided as aliases in CloudWatch Logs Insights. The query language documentation covers aliases using the as keyword in commands like fields, parse, sort, and stats, but does not mention reserved words or identifiers that might cause unexpected behavior.

Your workaround of using a different alias name is the correct approach. Given the systematic nature of your testing and the reproducibility across multiple log groups, this appears to be a query engine behavior that warrants reporting to AWS Support through the AWS Support Center at https://console.aws.amazon.com/support/. When you add correspondence about this issue, include your detailed reproduction steps, the specific queries, and the comparison results you've documented—this level of detail will help the service team investigate the root cause.

In the meantime, continue using alternative alias names like uid, current_user, or any other identifier that doesn't trigger this behavior. Your testing has already confirmed this produces correct results across all your log groups and time ranges.
Sources
Use aliases and comments in queries - Amazon CloudWatch Logs

answered a month ago

EXPERT

reviewed a month ago

  • I updated the question. I corrected some wrong findings.

  • This answer was generated for the first version of the question and no longer applies. Further testing showed the alias name is not the cause: uid, and even filter coalesce(user_id, userId) = "<id>" with no alias, drop the same rows. Recently ingested events were missed for several minutes, and some older time slices are also affected. Using a different alias name is not a reliable workaround. Filtering on raw fields (filter user_id = "<id>" or userId = "<id>") or @message like returned correct results in every test. See the edited question for the data.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.