How do I retrieve and analyze my CloudTrail logs with CloudWatch Logs Insights?

Short description

After you configure CloudTrail to send logs to a CloudWatch Logs log group, you can run queries against the log group. You can filter API activity by event type, user identity, AWS service, or time range.

This article provides sample queries to retrieve CloudWatch Logs to analyze and explore Amazon S3 bucket and object activity.

Note: By default, CloudTrail doesn't capture Amazon Simple Storage Service (Amazon S3) data events. You must turn on event logging in CloudTrail to retrieve event logs for Amazon S3 buckets and objects.

Resolution

Query CloudTrail logs with CloudWatch Logs Insights

Use the CloudWatch console to select the log group that receives your CloudTrail logs.

You can build on the following example queries to create additional and more complex Logs Insights queries that align to your use case. You can also integrate queries with your CloudWatch dashboard to visualize your queries as charts and graphs alongside related metrics.

Retrieve the most recent events

To retrieve recent CloudTrail Log events with the default @timestamp and @message fields, run the following example query:

#Retrieve the most recent CloudTrail events
fields @timestamp, @message
| sort @timestamp desc
| limit 2

Isolate individual fields

To isolate individual fields in the @message field and display selected fields in the CloudTrail event, run the following example query:

#Breakout Individual Fields
fields @timestamp, awsRegion, eventCategory, eventSource, eventName, eventType, sourceIPAddress, userIdentity.type
| sort @timestamp desc
| limit 2

The query returns columns such as @timestamp, awsRegion, eventCategory, eventSource, eventName, eventType, sourceIPAddress, and userIdentity.type.

Filter by Amazon EC2 RunInstances

Run the following query to retrieve the latest Amazon Elastic Compute Cloud (Amazon EC2) instances that launched in the AWS account based on the RunInstances API call:

#EC2: Recently Launched Instances
fields eventTime, eventName as API, responseElements.instancesSet.items.0.instanceId as InstanceID, userIdentity.sessionContext.sessionIssuer.type as IssuerType, userIdentity.type as IdentityType, userIdentity.sessionContext.sessionIssuer.userName as userName
| filter eventName = 'RunInstances'
| sort eventTime desc
| limit 2

Example output:

eventTime	API	InstanceID	IssuerType	IdentityType	userName
2022-02-18T17:36:38Z	RunInstances	i-0325b4d6ae4e93c75	Role	AssumedRole	AWSServiceRoleForAutoScaling

Filter by the most recent console login

Run the following query to retrieve the latest console logins based on the ConsoleLogin API call and to rename fields to use more meaningful labels:

#Console Login: Most Recent API Calls
fields eventTime, eventName, responseElements.ConsoleLogin as Response, userIdentity.arn as ARN, userIdentity.type as User_Type
| filter eventName = 'ConsoleLogin'
| sort eventTime desc
| limit 10

Example output:

eventTime	eventName	Response	User	User_Type
2022-02-18T17:35:44Z	ConsoleLogin	Success	arn:aws:iam::123456789012:user/test_user	IAMUser

Filter by the console login with failed authentication

Run the following query to retrieve unsuccessful console logins and to rename fields to use more meaningful labels:

#ConsoleLogin: Filter on Failed Logins
fields eventTime, eventName, responseElements.ConsoleLogin as Response, userIdentity.userName as User, userIdentity.type as User_Type, sourceIPAddress, errorMessage
| filter eventName = 'ConsoleLogin' and responseElements.ConsoleLogin = 'Failure'
| sort eventTime desc
| limit 10

Example output:

eventTime	eventName	Response	User	User_Type	sourceIPAddress	errorMessage
2022-02-18T20:10:55Z	ConsoleLogin	Failure	echo	IAMUser	12.34.56.89	Failed authentication

Filter by S3 object upload to a specific bucket

Run the following query to retrieve PutObject API calls to a bucket and to rename fields to use more meaningful labels:

#Filter PutObject API Calls on a specific S3 Bucket
fields @timestamp, eventName as API, requestParameters.bucketName as BucketName, requestParameters.key as Key, userIdentity.sessionContext.sessionIssuer.userName as UserName
| filter eventName = 'PutObject' and BucketName = 'target-s3-bucket'
| sort @timestamp desc
| limit 2

Note: Replace target-s3-bucket with the name of your bucket.

Example output:

@timestamp	API	Bucket_Name	Key	UserName
2022-02-12 17:16:07.415	PutObject	test_bucket1	w4r9Hg4V7g.jpg

Note: S3 object-level events require that you turn on CloudTrail data events for S3. For instructions, see Logging data events.

Summarize S3 activity

Run the following query to filter events from the S3 service, aggregate events by count, and divide the results by API call, bucket name, and key:

#S3 Activity: Bucket Key Details
filter eventSource = 's3.amazonaws.com'
| stats count(*) as Hits by eventName as API, requestParameters.bucketName as BucketName, requestParameters.key as Key
| sort Hits desc
| limit 5

Example output:

API	BucketName	Key	Hits
ListAccessPoints	44
GetBucketAcl	team1-ctrail-multi-region	27
GetObject	devsupport-prod	rdscr/individual/123456789012	18

Summarize AWS KMS decrypt activity

Run the following query to filter events from the AWS Key Management Service (AWS KMS) Decrypt API and aggregate events by AWS KMS key and user:

#KMS Decrypt Activity: Key User Details
fields resources.0.ARN as KMS_Key, userIdentity.sessionContext.sessionIssuer.userName as User
| filter eventSource='kms.amazonaws.com' and eventName='Decrypt'
| stats count(*) as Hits by KMS_Key, User
| sort Hits desc
| limit 2

Example output:

KMS_Key	User	Hits
arn:aws:kms:us-east-1:123456789012:key/03f2923d-e213-439d-92cf-cbb444bd85bd	AWSServiceRoleForConfig	12

Summarize API calls with errors

Run the following query to filter all events that contain an error code, aggregate events by count, and divide the results by AWS service, API call, and error code:

#Summarize API Calls with Errors
filter ispresent(errorCode)
| stats count(*) as Num_of_Events by eventSource as AWS_Service, eventName as API, errorCode
| sort Num_of_Events desc
| limit 5

Example output:

AWS_Service	API	errorCode	Num_of_Events
s3.amazonaws.com	GetBucketPublicAccessBlock	NoSuchPublicAccessBlockConfiguration	79

Summarize S3 API calls with error codes

Run following query to filter S3 events that contain an error code and aggregate events by error code and error message:

#S3: Summarize Error Codes
filter eventSource = 's3.amazonaws.com' and ispresent(errorCode)
| stats count(*) as Hits by errorCode, errorMessage
| sort Hits desc
| limit 5

Example output:

errorCode	errorMessage	Hits
AccessDenied	Access Denied	86
NoSuchBucketPolicy	The bucket policy does not exist	80

Summarize "AccessDenied" and "UnauthorizedOperation" API calls

Run the query to filter events with "AccessDenied" or "UnauthorizedOperation" error codes and aggregate events by AWS service, API call, and the AWS Identity and Access Management (IAM) identity type:

#Summarize AccessDenied/UnauthorizedOperation API Calls by AWS Service, API, IAM User
filter (errorCode='AccessDenied' or errorCode='UnauthorizedOperation')
| stats count(*) as NumberOfEvents by errorCode, eventSource as AWS_Service, eventName as API, userIdentity.type as IdentityType, userIdentity.invokedBy as InvokedBy
| sort NumberOfEvents desc
| limit 10

Example output:

errorCode	AWS Service	API	IdentityType	InvokedBy	NumberOfEvents
AccessDenied	s3.amazonaws.com	HeadBucket	AWSService	delivery.logs.amazonaws.com	83

Summarize AWS KMS hourly decrypt call volume

Run the following query to filter AWS KMS Decrypt API calls and aggregate events into one-hour bins.

#KMS: Hourly Decrypt Call Volume
filter eventSource='kms.amazonaws.com' and eventName='Decrypt'
| stats count(*) as Hits by bin(1h)

Example output:

bin(1h)	Hits
2022-02-18 19:00:00.000	16
2022-02-18 18:00:00.000	25

Related information

Monitor AWS CloudTrail log data in Amazon CloudWatch (Video)

Add a query to dashboard or export query results