OpenSearch ConnectionError: Max retries exceeded with url: /index_001/_search

Question

I'm getting the following error when attempting to run a search query against OpenSearch domain.

`"ConnectionError(HTTPSConnectionPool(host='search-osdomain7k309k4j-v82k34jcvhs-fvksd83874jflks43o.us-east-1.es.amazonaws.com', port=9200): Max retries exceeded with url: /index_001/_search (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known')))`

I also tried with port 443, same error.

Here is my code to establish an Opensearch client and run a query:

```
# OpenSearch
osHost = "search-osdomain7k309k4j-v82k34jcvhs-fvksd83874jflks43o.us-east-1.es.amazonaws.com"
osPort = 9200
osIndex = "index_001"

# OpenSearch client 
typeSearch = "_search"
region = "us-east-1"
service = "es"
credentials = boto3.Session().get_credentials()
awsauth = AWSV4SignerAuth(credentials, region, service)
osClient = OpenSearch(
    hosts=[{'host': osHost, 'port': osPort}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection,
    pool_maxsize=20
)

query = {
        'size': 123,
        'query': {
            'multi_match': {
                'query': "search phrase asdf",
                'fields': ['*'],
                'fuzziness': 'AUTO'
            }
        },
        'highlight': {
            'fields': {
                '*': {}
            }
        }
    }

response = osClient.search(
        index=osIndex,
        body=query
    )
```

The above code runs in a lambda. I believe networking issues can be ruled out as a potential cause, since I have another lambda with the same configuration, which uses an Opensearch client to bulk update the same OS domain I am trying to connect to in the above code.

Please advise on how I can connect to the OS domain to run a search query.

re:Post Agent · Answer

Based on the error message and the information provided, it appears that there might be a few issues to address in order to successfully connect to your OpenSearch domain and run a search query:

1. Endpoint URL: The error suggests that the hostname cannot be resolved. Double-check that the OpenSearch domain endpoint URL is correct. Make sure there are no typos in the domain name.

2. Port: You mentioned trying both port 9200 and 443. For Amazon OpenSearch Service, the standard HTTPS port 443 is typically used. Update your code to use port 443 instead of 9200.

3. VPC Configuration: If your OpenSearch domain is within a VPC, ensure that your Lambda function is also configured to run within the same VPC and has the necessary network access to reach the OpenSearch domain.

4. Security Group: If using a VPC, check that the security group associated with your OpenSearch domain allows inbound traffic from the Lambda function's security group.

5. IAM Permissions: Verify that the IAM role associated with your Lambda function has the necessary permissions to access the OpenSearch domain.

Here's a modified version of your code that addresses some of these issues:

```python
# OpenSearch
osHost = "search-osdomain7k309k4j-v82k34jcvhs-fvksd83874jflks43o.us-east-1.es.amazonaws.com"
osPort = 443  # Changed to 443
osIndex = "index_001"

# OpenSearch client 
region = "us-east-1"
service = "es"
credentials = boto3.Session().get_credentials()
awsauth = AWSV4SignerAuth(credentials, region)  # Removed 'service' parameter
osClient = OpenSearch(
    hosts=[{'host': osHost, 'port': osPort}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection,
    pool_maxsize=20
)

# ... rest of your code remains the same
```

If you're still encountering issues after making these changes, you may want to:

1. Use the AWS CLI or AWS Console to verify that your OpenSearch domain is in an "Active" state and accessible.
2. Check CloudWatch Logs for your Lambda function to see if there are any additional error messages or stack traces that could provide more insight into the connection issue.
3. Consider using the OpenSearch Service API or AWS SDK to test connectivity to your domain outside of your Lambda function to isolate whether the issue is specific to the Lambda environment.

If the problem persists, you may need to contact AWS support for further assistance, as there could be underlying issues with your OpenSearch domain or VPC configuration that are not immediately apparent from the provided information.  
**Sources**  
[Troubleshoot OpenSearch Service timeout issues | AWS re:Post](https://repost.aws/knowledge-center/opensearch-timeout-issues)  
[Troubleshooting Amazon OpenSearch Service - Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/handling-errors.html)

rePost-User-4029289 · Answer

Hello,

Warm Greetings, thank you for reaching to us with your query. Based on the error message and the information provided, it appears that there might be a few issues to address in order to successfully connect to your OpenSearch domain and run a search query:

- Endpoint URL: The error suggests that the hostname cannot be resolved. Double-check that the OpenSearch domain endpoint URL is correct. Make sure there are no typos in the domain name.

- Port: You mentioned trying both port 9200 and 443. For Amazon OpenSearch Service, the standard HTTPS port 443 is typically used. Update your code to use port 443 instead of 9200.

Here's a modified version of your code that addresses some of these issues:

# OpenSearch
osHost = "search-osdomain7k309k4j-v82k34jcvhs-fvksd83874jflks43o.us-east-1.es.amazonaws.com"
osPort = 443  # Changed to 443
osIndex = "index_001"

# OpenSearch client 
region = "us-east-1"
service = "es"
credentials = boto3.Session().get_credentials()
awsauth = AWSV4SignerAuth(credentials, region)  # Removed 'service' parameter
osClient = OpenSearch(
    hosts=[{'host': osHost, 'port': osPort}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection,
    pool_maxsize=20
)

# ... rest of your code remains the same

If you're still encountering issues after making these changes, you may want to:

1) Use the AWS CLI or AWS Console to verify that your OpenSearch domain is in an "Active" state and accessible.
    2) Try using the same code in an EC2 instance within the same VPC settings. It will help isolate if the issue is with Lambda or the code itself.
    3) Consider using the OpenSearch Service API or AWS SDK to test connectivity to your domain outside of your Lambda function to isolate whether the issue is specific to the Lambda environment.

If the problem persists, you may need to contact AWS support for further assistance, as there could be underlying issues with your OpenSearch domain or VPC configuration that are not immediately apparent from the provided information.

[+] https://docs.aws.amazon.com/opensearch-service/latest/developerguide/handling-errors.html
[+] https://docs.aws.amazon.com/opensearch-service/latest/developerguide/search-example.html

OpenSearch ConnectionError: Max retries exceeded with url: /index_001/_search

OpenSearch

OpenSearch client

... rest of your code remains the same

Relevant content