How do I manage the log levels of my AWS IoT logs in AWS IoT Core?

Short description

Note: The following resolution is for only AWS IoT Core logs v2.

As a best practice, know what AWS IoT Core resources you must monitor and at what level. Divide your AWS IoT Core resources, and then assign each segment a suitable log level. You can set default log levels or specific log levels for your resources.

For example, configure default logging to have a less verbose log level, such as ERROR or WARN. When you configure default logging, the configuration is applied to all resources that don't have resource-specific logging configured. To have a more verbose level, such as INFO or DEBUG, configure resource-specific logging. You can configure resource-level logging for different target types, such as THING_GROUP, CLIENT_ID, SOURCE_IP, or PRINCIPAL_ID.

You can simultaneously use default logging and resource-specific logging.

Important: Based on your AWS IoT Core fleet size, more verbose log levels can incur higher costs and can be more difficult to troubleshoot. More verbose log levels also create higher data traffic. It's a best practice to use INFO or DEBUG only to temporarily troubleshoot. After you troubleshoot, reset log levels to a less verbose level. For more information about log levels, see Log levels.

Resolution

Prerequisites:

• You installed the AWS Command Line Interface (AWS CLI) on your local machine with AWS IoT administrator permission credentials. The default AWS Region for AWS CLI must point toward the targeted Region.
• You have clients connected to your AWS IoT Core, either as registered or non-registered AWS IoT things.

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshoot AWS CLI errors. Also, make sure that you're using the most recent AWS CLI version.

Use default logging to lower costs and data traffic

When you set the default log level for your AWS IoT logs to ERROR or WARN, you can review log errors or warnings. These verbose levels lower your costs and your data traffic. For more information, see How do I configure the default logging settings for AWS IoT Core?

Note: If you create an AWS Support case, then you must provide AWS IoT Core logs.

Manually log resources

For information about how to manually log resources, see How do I configure logging levels manually for specific resources in AWS IoT Core?

Dynamically log resources

For more information about how to dynamically log resources, see How do I configure logging levels dynamically for specific resources in AWS IoT Core?

Monitor log entries

Use either the Amazon CloudWatch console or the AWS CLI to monitor your AWS IoT log entries.

CloudWatch console

To monitor your AWS IoT logs in the CloudWatch console, see Viewing AWS IoT logs in the CloudWatch console.

AWS CLI

To get your logs in the AWS CLI, run the filter-log-events command. Replace YourStartTimeInMilliseconds with your start time in milliseconds:

aws logs filter-log-events --log-group-name AWSIotLogsV2 \
 --start-time YourStartTimeInMilliseconds

Note: You can use the --filter-pattern to search for specific log attributes. For more information, see Filter and pattern syntax for metric filters, subscription filters, filter log events, and Live Tail.

The logLevel value is the log level that's configured for the target. In this example, the log level for the target is INFO:

{
 "timestamp": "2017-08-10 15:37:23.476",
 "logLevel": "INFO",
 "traceId": "20b23f3f-d7f1-feae-169f-82263394fbdb",
 "accountId": "123456789012",
 "status": "Success",
 "eventType": "Connect",
 "protocol": "MQTT",
 "clientId": "abf27092886e49a8a5c1922749736453",
 "principalId": "145179c40e2219e18a909d896a5340b74cf97a39641beec2fc3eeafc5a123456",
 "sourceIp": "123.123.123.123",
 "sourcePort": 13490
}

Note: CLIENT_ID is available only when you connect to AWS IoT Core with MQTT or MQTT over WSS protocols. HTTPS protocol doesn't support client IDs. Instead, use SOURCE_IP or PRINCIPAL_ID target types.

Edit or delete log levels

You can change the default log level and resource-specific log level at any time. It can take up to 10 minutes for the changes to complete.

Run the delete-v2-logging-level AWS CLI command. Replace YourTargetName with your target's name:

aws iot delete-v2-logging-level \
 --target-type "THING_GROUP | CLIENT_ID | SOURCE_IP | PRINCIPAL_ID" \
 --target-name YourTargetName

Related information

AWS IoT Device Management endpoints and quotas

Monitoring AWS IoT

AWS IoT Device Management pricing