I am trying to list the databases in my DocumentDb from a Lambda function, and the task times out. It is a simple command it should not be timing out. What do I need to fix so that the command runs properly?
client = MongoClient(connectionString)
logging.info("All the databases")
logging.info(client.list_database_names())
When I run the code on my VS Code IDE I get this message;
Exception has occurred: ServerSelectionTimeoutError
docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed, Timeout: 30s, Topology Description: <TopologyDescription id: 641e0297b555a6d973f82353, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed')>]>
File "C:\mma\Lambda_tweets\lambda_handler.py", line 47, in getDBConnection
logging.info(client.list_database_names())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\mma\Lambda_tweets\lambda_handler.py", line 127, in lambda_handler
getDBConnection()
File "C:\mma\Lambda_tweets\lambda_handler.py", line 130, in <module>
lambda_handler(None, None)
pymongo.errors.ServerSelectionTimeoutError: docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed, Timeout: 30s, Topology Description: <TopologyDescription id: 641e0297b555a6d973f82353, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed')>]>
EDIT: When I extended the timeout as suggested in the comments, the error messages I was getting were as follows;
Response
{
"errorMessage": "docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known, Timeout: 30s, Topology Description: <TopologyDescription id: 641f88d7e7919b310a30cf51, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known')>]>",
"errorType": "ServerSelectionTimeoutError",
"requestId": "2f9fb04f-9519-41e9-b989-0dd8b6f3382b",
"stackTrace": [
" File "/var/task/lambda_function.py", line 62, in lambda_handler\n getDBConnection()\n",
" File "/var/task/lambda_function.py", line 51, in getDBConnection\n logging.info(client.list_database_names())\n",
" File "/opt/python/pymongo/mongo_client.py", line 1867, in list_database_names\n return [doc["name"] for doc in self.list_databases(session, nameOnly=True, comment=comment)]\n",
" File "/opt/python/pymongo/mongo_client.py", line 1840, in list_databases\n res = admin._retryable_read_command(cmd, session=session)\n",
" File "/opt/python/pymongo/database.py", line 849, in _retryable_read_command\n return self.__client._retryable_read(_cmd, read_preference, session)\n",
" File "/opt/python/pymongo/_csot.py", line 105, in csot_wrapper\n return func(self, *args, **kwargs)\n",
" File "/opt/python/pymongo/mongo_client.py", line 1441, in _retryable_read\n server = self._select_server(read_pref, session, address=address)\n",
" File "/opt/python/pymongo/mongo_client.py", line 1257, in _select_server\n server = topology.select_server(server_selector)\n",
" File "/opt/python/pymongo/topology.py", line 272, in select_server\n server = self._select_server(selector, server_selection_timeout, address)\n",
" File "/opt/python/pymongo/topology.py", line 261, in _select_server\n servers = self.select_servers(selector, server_selection_timeout, address)\n",
" File "/opt/python/pymongo/topology.py", line 223, in select_servers\n server_descriptions = self._select_servers_loop(selector, server_timeout, address)\n",
" File "/opt/python/pymongo/topology.py", line 238, in _select_servers_loop\n raise ServerSelectionTimeoutError(\n"
]
}
Function Logs
START RequestId: 2f9fb04f-9519-41e9-b989-0dd8b6f3382b Version: $LATEST
[INFO] 2023-03-25T23:50:47.126Z 2f9fb04f-9519-41e9-b989-0dd8b6f3382b get client
/opt/python/pymongo/common.py:813: UserWarning: Unknown option ssl_ca_certs
warnings.warn(str(exc))
[INFO] 2023-03-25T23:50:47.380Z 2f9fb04f-9519-41e9-b989-0dd8b6f3382b Get database testdb
[INFO] 2023-03-25T23:50:47.380Z 2f9fb04f-9519-41e9-b989-0dd8b6f3382b All the databases
[ERROR] ServerSelectionTimeoutError: docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known, Timeout: 30s, Topology Description: <TopologyDescription id: 641f88d7e7919b310a30cf51, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known')>]>
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 62, in lambda_handler
getDBConnection()
File "/var/task/lambda_function.py", line 51, in getDBConnection
logging.info(client.list_database_names())
File "/opt/python/pymongo/mongo_client.py", line 1867, in list_database_names
return [doc["name"] for doc in self.list_databases(session, nameOnly=True, comment=comment)]
File "/opt/python/pymongo/mongo_client.py", line 1840, in list_databases
res = admin._retryable_read_command(cmd, session=session)
File "/opt/python/pymongo/database.py", line 849, in _retryable_read_command
return self.__client._retryable_read(_cmd, read_preference, session)
File "/opt/python/pymongo/_csot.py", line 105, in csot_wrapper
return func(self, *args, **kwargs)
File "/opt/python/pymongo/mongo_client.py", line 1441, in _retryable_read
server = self._select_server(read_pref, session, address=address)
File "/opt/python/pymongo/mongo_client.py", line 1257, in _select_server
server = topology.select_server(server_selector)
File "/opt/python/pymongo/topology.py", line 272, in select_server
server = self._select_server(selector, server_selection_timeout, address)
File "/opt/python/pymongo/topology.py", line 261, in _select_server
servers = self.select_servers(selector, server_selection_timeout, address)
File "/opt/python/pymongo/topology.py", line 223, in select_servers
server_descriptions = self._select_servers_loop(selector, server_timeout, address)
File "/opt/python/pymongo/topology.py", line 238, in _select_servers_loop
raise ServerSelectionTimeoutError(END RequestId: 2f9fb04f-9519-41e9-b989-0dd8b6f3382b
REPORT RequestId: 2f9fb04f-9519-41e9-b989-0dd8b6f3382b Duration: 30494.77 ms Billed Duration: 30495 ms Memory Size: 128 MB Max Memory Used: 66 MB Init Duration: 555.69 ms
The point is that the command is a simple command and it should not be tiiming out. So why is it timing out?
Yes it's a simple command but it's making a connection to a database, and that connection might not be configured properly and could be on the way to timing out. I see in your info "Timeout: 30s" is mentioned. Try extending your Lambda function's timeout to e.g. 5 minutes which will give enough time for your DB connection attempt to complete one way or another, then it will be easier to debug, especially if you trap and fully log errors.
I can now appreciate your reply @skinsman. I got more error details which I posted back in the question. I am not sure what they mean. Do I need to sort out my ssl_ca_certs?
That's getting into DocumentDB-specifics, and I'm no expert there. A quick Google of "Name or service not known" found something that may help though - https://stackoverflow.com/questions/65222660/connecting-to-documentdb-from-aws-lambda-using-python
As far as I can tell I have followed the steps to the letter but I still have this problem. Is there someone who can help me with this? I really can't do anything until this is fixed. Or I guess I can use another database, maybe DynamoDB.