How do I troubleshoot Kibana or OpenSearch Dashboards errors in OpenSearch Service?

Resolution

If there are underlying infrastructure or configuration issues, then you receive one of the following error messages:

"OpenSearch Dashboards/Kibana server is not ready yet"

"OpenSearch Dashboards/Kibana did not load properly"

"OpenSearch Dashboards/Kibana did not load properly. Check the server output for more information."

To troubleshoot these issues and verify that your server works correctly, check the following settings:

• Make sure that your CPU utilization doesn't exceed 80%.
• Make sure that your Java Virtual Machine (JVM) memory pressure metric doesn't exceed 75%.
• Troubleshoot ClusterBlockException and IndexCreateBlockException errors that you receive.
• Check for low storage space on your OpenSearch Service domain.
• Evenly distribute shards across nodes.
• Make sure that shards don't exceed the default threshold value of 1000 in each node. To decrease the shard count for your nodes, delete unneeded indexes or add more nodes to your cluster. To delete indexes, see Delete index API on the OpenSearch website. For more information about shard best practices, see Choosing the number of shards.
• Check for failed nodes.
• Don't use bulk queries that create a heavy load on your cluster. If you must use bulk queries to improve indexing performance, then start with a small bulk size and incrementally increase it. For more information, see Using and sizing bulk requests on the Elastic website.
• Confirm that the OpenSearch Service domain's resource-based policy allows AWS Identity and Access Management (IAM) users and roles to interact with the service.
  Note: You can also use an IP-based policy. You can't use an identity-based policy to configure access to the Kibana dashboard or OpenSearch Dashboards.

If you still encounter issues, then use the following troubleshooting steps to resolve your issue.

Troubleshoot the "Http request timed out" error or VPC access issues

If your OpenSearch Service cluster is in a virtual private cloud (VPC), then you might receive the "{"Message":"Http request timed out connecting"}" error message. This issue occurs when a network reachability issue blocks connection to Kibana or OpenSearch Dashboards.

To troubleshoot this issue, take the following actions:

• Make sure that the VPC's security groups allow inbound traffic on port 443 from your client IP address.
• If your client is in a different VPC than your OpenSearch Service cluster, then confirm that you correctly configured the client and OpenSearch Service subnets. For example, check your VPC endpoints, VPC peering connection, VPN, or AWS Direct Connect connection.
• Make sure that your subnet route tables have the necessary routes to access the OpenSearch Service domain.

For more information, see Launching your OpenSearch Service domains within a VPC.

Troubleshoot the "User: anonymous is not authorized to perform" error

If you activated fine-grained access control and use SAML 2.0 authentication, then you might receive the following error message:

"{"Message":"User: anonymous is not authorized to perform: es:ESHttpGet because no resource-based policy allows the es:ESHttpGet action"}"

To troubleshoot this issue, take the following actions:

• Verify that your resource-based policy includes the necessary permissions for the authenticated SAML 2.0 users.
• Verify that you correctly mapped the backend roles to the SAML 2.0 attributes.

Troubleshoot SAML 2.0, Amazon Cognito, or IAM Identity Center integration issues

Take the following actions:

• Verify that you correctly configured the SAML 2.0 metadata in both your identity provider (IdP) and OpenSearch Service.
• Verify that you correctly set up and linked the Amazon Cognito user pool and identity pool to your OpenSearch Service domain. To troubleshoot issues, see Common configuration issues.
• Make sure that you configured the required IAM roles and permissions for Amazon Cognito authentication.

Troubleshoot custom proxy issues

If you experience issues when you use a custom proxy, then take the following actions:

• Use the endpoint URL to access Kibana or OpenSearch Dashboards without the proxy to check whether the proxy configuration blocks your access.
• Verify that your proxy forwards requests to the OpenSearch Service domain.
• Make sure that your proxy keeps the necessary headers. For information about how to configure the NGINX proxy, see How do I use an NGINX proxy to access Kibana or OpenSearch Dashboards outside of a VPC that doesn't use Amazon Cognito authentication?

For a sample proxy configuration, see Build SAML identity federation for OpenSearch Service domains within a VPC.

For Amazon Cognito authentication in a VPC, see How do I use an NGINX proxy to access OpenSearch Dashboards with Amazon Cognito authentication from outside a VPC?

Troubleshoot expired certificates

Expired certificates in proxy endpoints can cause access issues.

To troubleshoot access issues, take the following actions:

• Check the expiration dates of the certificates that you use in your proxy setup.
• Renew expired certificates, and update them in your proxy configuration.
• Monitor your certificates so that you're notified before the certificate expires.

Troubleshoot the "exceeded the number of permissible concurrent requests" error or throttled resource issues

If your nodes receive too many signed requests with different IAM credentials, then you receive the following error message:

"You have exceeded the number of permissible concurrent requests with unique IAM Identities"

To troubleshoot this issue, take the following actions:

• Consolidate your IAM roles or users to reduce the variety of credentials that are used for requests.
• Add nodes to your cluster to increase the node-level throttling thresholds.
• Implement retry logic in your client applications.
• Gradually increase the request volume.
• Avoid bulk operations with multiple credentials.

Troubleshoot index and migration issues during blue/green deployments

You might experience issues during blue/green deployments because of the following issues:

• Wildcard index pattern template conflicts that affect the Kibana or OpenSearch Dashboards index.
• Corrupted documents in the Kibana or OpenSearch Dashboards index.

To resolve these issues, run the following command to check for conflicting index templates:

GET _cat/templates

Example output:

GET _cat/templates?v

name                   index_patterns order      version composed_of
my-index-template      [*] 0

GET _template/my-index-template

{
  "my-index-template": {
    "order": 0,
    "index_patterns": [
      "*"
    ],
    "settings": { <index-settings> },
    "mappings": { <index-mappings> },
    "aliases": { <index-alias> }
  }
}

For more information, see CAT templates API on the OpenSearch website.

It isn't a best practice to use "index_patterns": "*" in your index template because it includes all indexes. Instead, run the following command to modify the index template to include only indexes with a specific prefix:

PUT _template/my-index-template
{
    "index_patterns": ["your-index-prefix-*"],
    "settings": { index-settings },
    "mappings": { index-mappings },
    "aliases": { index-alias }
}

Note: Replace your-index-prefix-* with an index prefix, index-settings with your index settings, index-mappings with your index mappings, and index-alias with your index alias. For more information, see Put template on the OpenSearch website.

If needed, then delete the Kibana or OpenSearch Dashboards index. For more information, see Delete index API on the OpenSearch website. Then, restore the index from a manual snapshot.

Troubleshoot single node failures

Individual node crashes can cause you to lose access to Kibana or OpenSearch Dashboards.

To troubleshoot this issue, take the following actions:

• Check the cluster metrics in Amazon CloudWatch for the affected nodes to identify when they become unhealthy:
  For Elasticsearch version 7.10 and earlier, check KibanaHealthyNodes.
  For OpenSearch version 1.x and later, check OpenSearchDashboardsHealthyNodes.
• Increase the number of nodes in your cluster for better redundancy.

If you have a single node, then complete the following steps to restore the .kibana* index from automated snapshots:

1. To list the automated snapshots available in your repository, run the following command:

   GET /_cat/snapshots/repository

   Note: Replace repository with cs-automated for a non-encrypted domain and cs-automated-enc for an encrypted domain.
2. To view a list of the indexes in a specific snapshot, run the following command:

   GET _snapshot/repo-name/snapshot-id?pretty

   Note: Replace repo-name with your repository name and snapshot-id with the snapshot ID.
3. To restore the .kibana* index, run the following command:

   POST _snapshot/repo-name/snapshot-id
   { "indices": ".kibana*" }

   Note: Replace repo-name with your repository name and snapshot-id with the snapshot ID. If you experience permissions issues when you try to restore the index, then open an AWS Support case. Both Kibana and OpenSearch Dashboards use the kibana* index.

For more information, see Why did my OpenSearch Service node crash?

Troubleshoot the "OpenSearch Security not initialized" error

The "OpenSearchSecurityException[OpenSearch Security not initialized for __PATH__]" error message occurs when security initialization fails.

To verify that the security plug-in initialized for the domain, open the https://kibana-dashboards-url/api/status?pretty URL in your browser.

Note: Replace kibana-dashboards-url with your dashboard URL.

Example output:

... Redacted ...
      {
        "id": "plugin:securityDashboards@2.15.0",
        "message": "All dependencies are available",
        "since": "2025-08-22T11:23:58.243Z",
        "state": "green",
        "icon": "success",
        "uiColor": "secondary"
      },
... Redacted ...

If the state isn't green in the output, then reopen the URL. If the state still isn't green or fluctuates each time that you open the URL, then the security plug-in isn't initialized correctly. To troubleshoot this issue, open an AWS Support case.

Troubleshoot "500" errors

If your CPU utilization or JVM memory pressure are high, then you might receive the following error message:

"{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}"

Make sure that your CPU utilization and JVM memory pressure aren't too high. For more information about the issues that you encounter, check the OpenSearch logs for more information about the errors or exception. If CPU utilization and JVM memory pressure are low but you still receive the "500" error, then open an AWS Support case.

Troubleshoot information that's missing

You might receive one of the following error messages:

• "OpenSearch Dashboards/Kibana blank page"
• "Unable to see any data on OpenSearch Dashboards/Kibana Visualization or Dashboards"

If there's missing information on the Discovery page of the dashboard, then complete the following steps to verify that your documents have a timestamp field:

1. Open Kibana or OpenSearch Dashboards.
2. Choose Dashboards Management.
3. Choose Index Pattern.
4. Select your index pattern.
5. For Time field, select Timestamp.

After you configure a timestamp field for your documents, check the following configurations:

• To verify that you correctly configured the timestamp field, run the following command in the Dev Tools console:

  GET your-index/_mappings?pretty

  Note: Replace your-index with your index name. In the output, make sure that timestamp is set to date. If it isn't set to date, then use the _reindex API to set the timestamp. For more information, see Reindex data on the OpenSearch website.
• Make sure that you selected the correct time range for your index pattern on the Discover page.
• If you activated fine-grained access control, then verify that the backend role has permissions to view the data.

Monitor the health status of your nodes

Kibana and OpenSearch Dashboards use the same CPUUtilization and JVMMemoryPressure resources as the OpenSearch Service nodes. To check the health status of your nodes, review the KibanaHealthyNodes or OpenSearchDashboardsHealthyNodes metric. Or, check the Kibana Health status on your domain's Cluster health tab on the OpenSearch Service console.

The health status metrics show one of the following states of the Kibana or OpenSearch Dashboards plug-in that runs on your data nodes:

• Green shows that Kibana or OpenSearch Dashboards is running on all data nodes and there are no known issues.
• Yellow shows that Kibana or OpenSearch Dashboards is running on some, but not all, of the data nodes. Kibana or OpenSearch Dashboards runs slowly in this status.
• Red shows that Kibana or OpenSearch Dashboards is down on all nodes. Kibana or OpenSearch Dashboards doesn't run in this status and users don't have access.