MongoDB Atlas Connection - AWS Glue

0

When I'm trying to import a collection from MongoDB Atlas to AWS Glue using a MongoDB connection, I get the following error :

Session failure. InvalidInputException - Unable to resolve any valid connection (Service: AWSGlueJobExecutor; Status Code: 400; Error Code: InvalidInputException; Request ID: 2e7885ad-a3d4-460c-a876-9921f905830d; Proxy: null) Error details Request id: 3ef5ee61-7035-4adb-a527-e49f5069341a Status code: 400

I have followed the documentation regarding the url :

Connection URL : mongodb+srv://proj1.z1qzqz6.mongodb.net/MVP1 Database : MVP1 Collection : Projects Partition size (MB) : 10 Partition key : _id

In MongoDB I have whiytelisted all IP addresses and I have confirmed that I'm using the right username and password in my connection.

Marc
asked 2 months ago162 views
1 Answer
0

Session failure. InvalidInputException - Unable to resolve any valid connection (Service: AWSGlueJobExecutor; Status Code: 400; Error Code: InvalidInputException; Request ID: 2e7885ad-a3d4-460c-a876-9921f905830d; Proxy: null) Error details Request id: 3ef5ee61-7035-4adb-a527-e49f5069341a Status code: 400 This typically means that the information or parameters provided in the request did not meet the expected format or values required by AWS Glue.

When attempting to access a collection within MongoDB Atlas, your configuration settings should resemble the following:

mongo_uri = "mongodb+srv://proj1.z1qzqz6.mongodb.net:27017/MVP1"

read_mongo_options = {
    "uri": mongo_uri,
    "database": "MVP1",
    "collection": "Projects",
    "username": "mongodbUsername",
    "password": "mongodbPassword",
    "partitioner": "MongoSamplePartitioner",
    "partitionerOptions": {
        "partitionSizeMB": "10",
        "partitionKey": "_id"
    }
}

Additionally, ensure that your Glue permission policy includes the glue:GetJobRun* action to maintain proper access and functionality.

profile picture
EXPERT
answered 2 months ago
  • Hello Osvaldo,

    Thank you for your reply.

    I still get the same error.

    This is my current configuration :

    MongoDB_node1708273422672 = glueContext.create_dynamic_frame.from_options( connection_type="mongodb", connection_options={ "uri": "mongodb+srv://proj1.z1qzqz6.mongodb.net:27017/MVP1", "database": "MVP1", "collection": "Projects", "username": "mongodbUsername", "password": "mongodbPassword", "partitioner": "MongoSamplePartitioner", "partitionerOptions": { "partitionSizeMB": "10", "partitionKey": "_id" } }, transformation_ctx="MongoDB_node1708273422672", )


    My policy has : "glue:*", so I should have proper access and functionality.

    Thank you very much for your help, Marc

  • I've identified the problem, Marc. When using glueContext.create_dynamic_frame.from_options, the connection type is limited to supporting only the following sources: S3, MySQL, PostgreSQL, Redshift, SQL Server, Oracle, and DynamoDB.

    That doesn't support MongoDB

    Resources:

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions