Questions tagged with Amazon DocumentDB

I am trying to list the databases in my DocumentDb from a Lambda function, and the task times out. It is a simple command it should not be timing out. What do I need to fix so that the command runs properly? ``` client = MongoClient(connectionString) logging.info("All the databases") logging.info(client.list_database_names()) ``` When I run the code on my VS Code IDE I get this message; > Exception has occurred: ServerSelectionTimeoutError docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed, Timeout: 30s, Topology Description: <TopologyDescription id: 641e0297b555a6d973f82353, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed')>]> File "C:\mma\Lambda_tweets\lambda_handler.py", line 47, in getDBConnection logging.info(client.list_database_names()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\mma\Lambda_tweets\lambda_handler.py", line 127, in lambda_handler getDBConnection() File "C:\mma\Lambda_tweets\lambda_handler.py", line 130, in <module> lambda_handler(None, None) pymongo.errors.ServerSelectionTimeoutError: docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed, Timeout: 30s, Topology Description: <TopologyDescription id: 641e0297b555a6d973f82353, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed')>]> EDIT: When I extended the timeout as suggested in the comments, the error messages I was getting were as follows; > Response { "errorMessage": "docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known, Timeout: 30s, Topology Description: <TopologyDescription id: 641f88d7e7919b310a30cf51, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known')>]>", "errorType": "ServerSelectionTimeoutError", "requestId": "2f9fb04f-9519-41e9-b989-0dd8b6f3382b", "stackTrace": [ " File \"/var/task/lambda_function.py\", line 62, in lambda_handler\n getDBConnection()\n", " File \"/var/task/lambda_function.py\", line 51, in getDBConnection\n logging.info(client.list_database_names())\n", " File \"/opt/python/pymongo/mongo_client.py\", line 1867, in list_database_names\n return [doc[\"name\"] for doc in self.list_databases(session, nameOnly=True, comment=comment)]\n", " File \"/opt/python/pymongo/mongo_client.py\", line 1840, in list_databases\n res = admin._retryable_read_command(cmd, session=session)\n", " File \"/opt/python/pymongo/database.py\", line 849, in _retryable_read_command\n return self.__client._retryable_read(_cmd, read_preference, session)\n", " File \"/opt/python/pymongo/_csot.py\", line 105, in csot_wrapper\n return func(self, *args, **kwargs)\n", " File \"/opt/python/pymongo/mongo_client.py\", line 1441, in _retryable_read\n server = self._select_server(read_pref, session, address=address)\n", " File \"/opt/python/pymongo/mongo_client.py\", line 1257, in _select_server\n server = topology.select_server(server_selector)\n", " File \"/opt/python/pymongo/topology.py\", line 272, in select_server\n server = self._select_server(selector, server_selection_timeout, address)\n", " File \"/opt/python/pymongo/topology.py\", line 261, in _select_server\n servers = self.select_servers(selector, server_selection_timeout, address)\n", " File \"/opt/python/pymongo/topology.py\", line 223, in select_servers\n server_descriptions = self._select_servers_loop(selector, server_timeout, address)\n", " File \"/opt/python/pymongo/topology.py\", line 238, in _select_servers_loop\n raise ServerSelectionTimeoutError(\n" ] } > Function Logs START RequestId: 2f9fb04f-9519-41e9-b989-0dd8b6f3382b Version: $LATEST [INFO] 2023-03-25T23:50:47.126Z 2f9fb04f-9519-41e9-b989-0dd8b6f3382b get client /opt/python/pymongo/common.py:813: UserWarning: Unknown option ssl_ca_certs warnings.warn(str(exc)) [INFO] 2023-03-25T23:50:47.380Z 2f9fb04f-9519-41e9-b989-0dd8b6f3382b Get database testdb [INFO] 2023-03-25T23:50:47.380Z 2f9fb04f-9519-41e9-b989-0dd8b6f3382b All the databases [ERROR] ServerSelectionTimeoutError: docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known, Timeout: 30s, Topology Description: <TopologyDescription id: 641f88d7e7919b310a30cf51, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known')>]> Traceback (most recent call last):   File "/var/task/lambda_function.py", line 62, in lambda_handler     getDBConnection()   File "/var/task/lambda_function.py", line 51, in getDBConnection     logging.info(client.list_database_names())   File "/opt/python/pymongo/mongo_client.py", line 1867, in list_database_names     return [doc["name"] for doc in self.list_databases(session, nameOnly=True, comment=comment)]   File "/opt/python/pymongo/mongo_client.py", line 1840, in list_databases     res = admin._retryable_read_command(cmd, session=session)   File "/opt/python/pymongo/database.py", line 849, in _retryable_read_command     return self.__client._retryable_read(_cmd, read_preference, session)   File "/opt/python/pymongo/_csot.py", line 105, in csot_wrapper     return func(self, *args, **kwargs)   File "/opt/python/pymongo/mongo_client.py", line 1441, in _retryable_read     server = self._select_server(read_pref, session, address=address)   File "/opt/python/pymongo/mongo_client.py", line 1257, in _select_server     server = topology.select_server(server_selector)   File "/opt/python/pymongo/topology.py", line 272, in select_server     server = self._select_server(selector, server_selection_timeout, address)   File "/opt/python/pymongo/topology.py", line 261, in _select_server     servers = self.select_servers(selector, server_selection_timeout, address)   File "/opt/python/pymongo/topology.py", line 223, in select_servers     server_descriptions = self._select_servers_loop(selector, server_timeout, address)   File "/opt/python/pymongo/topology.py", line 238, in _select_servers_loop     raise ServerSelectionTimeoutError(END RequestId: 2f9fb04f-9519-41e9-b989-0dd8b6f3382b REPORT RequestId: 2f9fb04f-9519-41e9-b989-0dd8b6f3382b Duration: 30494.77 ms Billed Duration: 30495 ms Memory Size: 128 MB Max Memory Used: 66 MB Init Duration: 555.69 mslg...

I am trying to get a list of database names for my DocumentDB. I am using the pymongo library. I run this code; ``` logging.info("get client") #connectionString = os.environ["documentdb_connection"] connectionString = "mongodb://geoff:qzmpqzmp@docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017/?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false" ##Specify the database to be used logging.info("Get database testdb") #db = client.testdb logging.info("All the databases") #db = client.testdb logging.info(client.list_database_names()) ``` I have been unable to connect to the specific database I wanted so I thought I would comment those lines out and instead list all of the databases to check if it exists. On the last line of that code I get a NameError; "name 'client' is not defined" How do I fix this?lg...

Hello, i create an cluster in DocumentDB and migrate all mongodb in our local servers. Our applications that use mongodb do not use autentacion, we handle connections using security groups in ec2. Is it possible to disable authentication to the documentdb? cluster if yes, as i do this?? Regards,lg...

I deleted and re-created the default VPC, but DocumentDB *default subnet group* still points to the deleted *VPC ID*: \ https://eu-west-1.console.aws.amazon.com/docdb/home?region=eu-west-1#subnetGroup-details/default How do I fix it?lg...

Starting from 2023-03-01 the `mongodump` command fails to exec on documentDB 4.0 instances. Before this date the `mongodump` command was working just fine. Error trace: **Failed: error checking for AtlasProxy: Unknown admin command atlasVersion** I've used mongodb-org-tools=4.4.15 and mongodb-org-tools=5.0.15 to run the `mongodump` with the same result. Command: ``` mongodump --ssl \ --host="$DB_HOST:$DB_PORT" \ --db="$DB_DATABASE" \ --username="$DB_USERNAME" \ --password="$DB_PASSWORD" \ --archive=./production-mongodump.gz \ --numParallelCollections=4 \ --authenticationDatabase="$DB_AUTHENTICATION_DATABASE" \ --authenticationMechanism=SCRAM-SHA-1 \ --sslCAFile=./rds-combined-ca-bundle.pem \ --gzip ``` Any ideas of what this can be?lg...

I am currently migrating my data from MongoDB to DocumentDB and want to use further use Documentdb as my permanent database. Now I first dumped the files and restored it on my documentDB cluster. I can also access the Documentdb cluster from mongoshell and I can see my database in Documentdb. Now my issue is that I am not able to use the database when I add the URI to my React application. It just doesn't work i tried several ways but it still didn't work. What can be the issue? I even followed the documentation.lg...

I created a DocumentDB cluster that is now connected to a few EC2s in a ASG. I know that for security reasons this should stay in a private subnet, is that really important? How can I check if my cluster is in a public subnet or a private one? And if it is in a public subnet, how can I move it to a private one? thanks Mlg...

Hi guys, we want to use DMS to migrate a MongoDB (on-prem) to DocumentDB (full load + CDC). The MongoDB is a "shared cluster" of which we use a mongos instance endpoint. One of 8 collections called "XXX" has over 300,000 documents with about 250K each. DMS runs well so far, but when loading the "XXX" collection the full load is aborted after approx. 10 - 15 minutes. CloudWatch Logs: [SOURCE_UNLOAD ]E: Failed reading from cursor for collection XXX: 'Connection closed by peer' [1028101] (mongodb_unload.c:179) [SOURCE_UNLOAD ]E: Failed while unloading collection 'XXX' [1028101] (mongodb_unload.c:208) [TASK_MANAGER ]W: Table 'mymongodb'.'Snapshots' (subtask 2 thread 0) is suspended (replicationtask.c:2550) We have already tried various variants for the "FullLoadSettings" in the migration task, unfortunately without success. With this setting, the termination is aborted after 15 minutes: ""FullLoadSettings"": { ""TargetTablePrepMode"": ""DROP_AND_CREATE"", ""CreatePkAfterFullLoad"": false, ""StopTaskCachedChangesApplied"": false, ""StopTaskCachedChangesNotApplied"": false, ""MaxFullLoadSubTasks"": 8, ""TransactionConsistencyTimeout"": 600, ""CommitRate"": 10000" Is there any way to control the connection time from DMS (to prevent "Connection closed by peer") or is this an known issue of MongoDB as a source? Any ideas are welcome!lg...

Hello, I am currently trying to migrate from a MongoDB Atlas cluster to Document DB. I am having issues with a DMS Source Endpoint connection. I am getting an error regarding connecting to the cluster using the Test Connection: `Test Endpoint failed: Application-Status: 1020912, Application-Message: Failed to create new client connection Failed to connect to database., Application-Detailed-Message: Error verifying connection: 'No suitable servers found (`serverSelectionTryOnce` set): [Server closed connection. calling ismaster on 'next-shard-00-01.xxxxxx.mongodb.net:27017']'` When I try to use mongosh or Mongo Compass I am able to connect with no issues. I changed some network settings in the Network Access section of Atlas to allow connections from everywhere. But still, it won't connect. Has anyone else had a successful migration from MongoDB Atlas to DocumentDB? The MongoDB version I'm using is 5.0.14 (API Version 1) DocumentDB is using Elastic Cluster setup. Thankslg...

i have a documentDb with, let's say **mycollection**, ewith docs such as ``` { "_id" : "id string 1", "field" : "Fieldvalue" "obj1" : { "until" : ISODate("2022-12-19T00:00:00Z") }, "obj2" : { "id" : 316524 }, "obj3" : { "status" : "available" }, "data" : { }, }, { "_id" : "id string 2", "field" : "Fieldvalue" "obj1" : { "until" : ISODate("2023-12-19T00:00:00Z"), id: 77, username: "some" }, "obj2" : { "id" : 316524 }, "obj3" : { "status" : "available" }, "data" : { }, }, ... ``` and respectively, i have an index ``` db.mycollection.createIndex( { "field": 1, "obj1.id": 1, "obj2.id": 1, "obj3.status": 1 }, { name: "my_special_index" } ) ``` but running ``` db.mycollection.updateMany( { $and: [ { field: "Fieldvalue" }, { "obj2.id": 24569 }, { "obj3.status": "available" }, { $or: [ { "obj1.until": { $lt: new Date()} }, { "obj1.id": 77 } ] } ] }, { $set: { "obj1": { "id": 77, "username": "some", "until": ISODate("2023-12-19T19:51:32.479Z") } } } ) ``` it takes **12 seconds**, that is way too much, given the whole collection is ~280k, and the records that match the condition ar 62k i have tried running explain on the **find()** ( unfortunately, no way to run explain on the updateMany() ) ``` db.mycollection.find( { $and: [ { field: "Fieldvalue" }, { "obj2.id": 24569 }, { "obj3.status": "available" }, { $or: [ { "obj1.until": { $lt: new Date()} }, { "obj1.id": 77 } ] } ] }, ).limit(100000).explain("executionStats").executionStats; ``` and i get, what are think is good response ``` { "executionSuccess" : true, "executionTimeMillis" : "173.373", "planningTimeMillis" : "1.064", "executionStages" : { "stage" : "SUBSCAN", "nReturned" : "60163", "executionTimeMillisEstimate" : "168.929", "inputStage" : { "stage" : "LIMIT_SKIP", "nReturned" : "60163", "executionTimeMillisEstimate" : "151.908", "inputStage" : { "stage" : "IXSCAN", "nReturned" : "60163", "executionTimeMillisEstimate" : "147.511", "indexName" : "my_special_index", "direction" : "forward" } } } } ``` i tried adding another index ``` db.mycollection.createIndex( { "field": 1, "obj1.id": 1, "obj1.until": -1, "obj2.id": 1, "obj3.status": 1}, { name: "my_special_index_with_until" } ) ``` but the **find().explain()** still prefers the first index, and **updateMany()** doesn't seem to run any faster so, my question is, what can i do to improve performance, i would expect these to run very fast Thanks!lg...

Dear all, First of all I'd like to ay that I'm new to DocuemntDB. With the help of https://docs.aws.amazon.com/documentdb/latest/developerguide/quick_start_cfn.html I've created a 3 instance DocumentDB with CloudFormation. The deployment finished without errors. I tried to connect with the root username and password I configured in the template without luck: ``` E QUERY [js] Error: Authentication failed. : ``` Then I tried without any user /password and I managed to connect: ``` $ mongo --ssl --host XYZ.docdb.amazonaws.com:27017 --sslCAFile rds-combined-ca-bundle.pem MongoDB shell version v4.0.28 connecting to: mongodb://XYZ.docdb.amazonaws.com:27017/admin?gssapiServiceName=mongodb Implicit session: session { "id" : UUID("11254a7a-6610-4db0-a545-3a8bc4e5cf99") } MongoDB server version: 4.0.0 rs0:PRIMARY> show dbs; rs0:PRIMARY> ``` As you can see there's nothing in the DB. Is it expected that CF cretaes such an empty DB? According to the connection string from the cluster descriptin I should be using ``` --username MyUserName --password <insertYourPassword> ``` My Code has nothing special: ``` DBCluster: Type: AWS::DocDB::DBCluster Properties: DeletionProtection: True BackupRetentionPeriod : 7 DBClusterIdentifier : !Ref DBClusterName DBClusterParameterGroupName : !Ref DBParameterGroup DBSubnetGroupName : !Ref DBSubnetGroup MasterUsername : MyUserName MasterUserPassword : !Ref DocDBSecret Port : "27017" PreferredBackupWindow : "23:00-23:59" PreferredMaintenanceWindow : "sun:00:00-sun:05:00" VpcSecurityGroupIds: - !Ref DBSecurityGroup StorageEncrypted : true EngineVersion: 4.0.0 ``` (I can paste other part of my template if needed) Is this behavior expected? TIAlg...

Hi, I have an application that connects to docDB and start listening to the changestream, but Performance Insights is showing high Database Load the moment we start watching the changestream, like shown above:   I would like to know the best way to read this chart. Apparently the changestream utilizes all CPUs from the instance, even when the database is idle and not receiving any queries. Do I need to worry about that high Database Load? Will enabling and reading the changestream make my database slower in general? What would be the effects on performance? AWS docs seen to be lacking on this information.lg...