How do I analyze AWS WAF logs in CloudWatch?

Resolution

To analyze and filter specific AWS WAF log requests in CloudWatch, use CloudWatch Logs Insights or the CloudWatch query generator.

CloudWatch Log Insights

You can use CloudWatch Log Insights from within the CloudWatch console or in Log Insights in AWS WAF.

AWS WAF

Complete the following steps:

1. Open the AWS WAF console.
2. In the navigation pane, choose AWS WAF.
3. Choose Resources & protection packs.
4. Select your Protection pack.
5. Choose view dashboard.
6. Expand Log explorer at the bottom of the page.
7. Choose View in CloudWatch.
8. In the Query editor, enter your query. Use query syntax to design your queries. You can also select queries from the Most frequently used queries list.
9. Choose Run query.

CloudWatch

Complete the following steps:

1. Open the CloudWatch console.
2. In the navigation pane, choose Logs.
3. Choose Log Insights.
4. Under log insights open the dropdown Selection criteria.
5. Select one or more log groups to query. Or, choose Browse log groups, and then select your log groups.
6. (Optional) Choose a time range for your query.
7. Use query syntax to design your queries.
8. To view your results, choose Run query.

CloudWatch query generator

To use generative AI to analyze your access logs, run the query generator in CloudWatch.

Example queries to search your data

To filter out specific information with CloudWatch Logs insights, use the following example queries.

Top client IPs

To count the top client IPs that access your logs, run the following query:

fields httpRequest.clientIp| stats count(*) as requestCount by httpRequest.clientIp
| sort requestCount desc

Top countries

To count the top countries that access your logs, run the following query:

stats count(*) as RequestCount by httpRequest.country as Country| sort RequestCount desc

Top hosts

To count the top hosts that access your logs, run the following query:

parse @message /\{"name":"[Hh]ost\",\"value":\"(?[^"}]*)/|stats count(*) as RequestCount by Host
| sort RequestCount desc

Top methods

To count the top methods that access your logs, run the following query:

stats count(*)as RequestCount by httpRequest.httpMethod as Method| sort RequestCount desc

Top user agents

To count the top user agents that access your logs, run the following query:

parse @message /\{"name":"[Uu]ser\-[Aa]gent\",\"value\"\:\"(?[^"}]*)/| stats count(*) as RequestCount by UserAgent
| sort RequestCount desc

Top terminating rules

To count the top terminating rules in your logs, run the following query:

stats count(*) as RequestCount by terminatingRuleId| sort RequestCount desc

Filter by blocked requests

To filter for all blocked requests and their terminating rule, URI path, and client IP, run the following query:

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule| filter action = "BLOCK"
| sort @timestamp desc

Filter by host

To filter your logs by a specific host, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.uri as URI| parse @message /\{"name":"[Hh]ost\",\"value":\"(?[^"}]*)/
| filter Host = "www.example.com"

Note: Replace www.example.com with the name of your host.

Filter by a specific string

To filter your logs by a specific string, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.httpMethod as Method,httpRequest.uri as URI| parse @message /\{"name":"[Hh]ost\",\"value":\"(?[^"}])/
| parse @message /\{"name":"[Uu]ser\-[Aa]gent\",\"value\"\:\"(?[^"}])/
| filter @message like "{jndi:ldap"  
| sort action, URI desc

Note: Replace {jndi:ldap with your string.

Filter by POST requests

To filter for POST requests, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.httpMethod as Method, httpRequest.uri as URI| parse @message /\{"name":"[Uu]ser\-[Aa]gent\",\"value\"\:\"(?[^"}]*)/| parse @message /\{"name":"[Hh]ost\",\"value":\"(?[^"}]*)/
| filter httpRequest.httpMethod ="POST"
| display Rule, action, Country, ClientIP, Method, URI, Host, UserAgent
| sort Rule, action desc

Filter by country

To filter out requests that don't originate from a specific country, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.uri as URI| parse @message /\{"name":"[Hh]ost\",\"value":\"(?[^"}]*)/
| parse @message /\{"name":"[Uu]ser\-[Aa]gent\",\"value\"\:\"(?[^"}]*)/
| filter Country != "US"
| sort Country, action desc

Note: Replace US with the country code that you want to filter out.

Filter by requests blocked by rate-based rules

To filter your logs that are blocked by a rate-based rule, run the following query:

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule, httpRequest.country as Country| filter action = "BLOCK"
| filter terminatingRuleType = "RATE_BASED"
| sort @timestamp desc

Cross-Site Scripting (XSS) or SQL Injection

To find patterns that cause XSS or SQL Injection in the terminating rule for custom rules or AWS Managed Rule Groups, run the following query:

fields @timestamp| parse @message ',"terminatingRuleMatchDetails":[*],' as terminatingRuleMatchData
| filter (terminatingRuleMatchData like /XSS/ or terminatingRuleMatchData like /SQL/)
| display @timestamp, httpRequest.clientIp, httpRequest.country, terminatingRuleMatchData, httpRequest.requestId

The query shows entries with a timestamp, client IP address, country of origin, match details, and the request ID.

Filter by requests counted by a specific rule in a rule group

To filter log entries for requests that a specific rule in a rule group counts and terminates by default, run the following query:

fields @timestamp| filter (@message like 'excludedRules":[{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"NoUserAgent_HEADER"}]}' and @message like 'terminatingRuleId":"Default_Action"')
| parse @message '"ruleId":*}]}' as ruleMatchDetails
| display @timestamp, httpRequest.clientIp, httpRequest.country, ruleMatchDetails, httpRequest.requestId

Note: Replace ruleId with your rule ID.

Filter by requests with a CAPTCHA that's not valid

To filter for the top 100 requests with a CAPTCHA that's not valid, run the following query:

fields @timestamp, httpRequest.clientIp, httpRequest.requestId, captchaResponse.failureReason, @message| filter captchaResponse.failureReason ='TOKEN_MISSING'
| sort @timestamp desc
| limit 100

Note: Replace 100 in the limit clause with the number of requests that you want to filter for.

This query shows the time of request, the IP address, the request ID, the response code, and the entire message.