MongoDB Atlas Connection - AWS Glue

0

When I'm trying to import a collection from MongoDB Atlas to AWS Glue using a MongoDB connection, I get the following error :

Session failure. InvalidInputException - Unable to resolve any valid connection (Service: AWSGlueJobExecutor; Status Code: 400; Error Code: InvalidInputException; Request ID: 2e7885ad-a3d4-460c-a876-9921f905830d; Proxy: null) Error details Request id: 3ef5ee61-7035-4adb-a527-e49f5069341a Status code: 400

I have followed the documentation regarding the url :

Connection URL : mongodb+srv://proj1.z1qzqz6.mongodb.net/MVP1 Database : MVP1 Collection : Projects Partition size (MB) : 10 Partition key : _id

In MongoDB I have whiytelisted all IP addresses and I have confirmed that I'm using the right username and password in my connection.

Marc
gefragt vor 3 Monaten177 Aufrufe
1 Antwort
0

Session failure. InvalidInputException - Unable to resolve any valid connection (Service: AWSGlueJobExecutor; Status Code: 400; Error Code: InvalidInputException; Request ID: 2e7885ad-a3d4-460c-a876-9921f905830d; Proxy: null) Error details Request id: 3ef5ee61-7035-4adb-a527-e49f5069341a Status code: 400 This typically means that the information or parameters provided in the request did not meet the expected format or values required by AWS Glue.

When attempting to access a collection within MongoDB Atlas, your configuration settings should resemble the following:

mongo_uri = "mongodb+srv://proj1.z1qzqz6.mongodb.net:27017/MVP1"

read_mongo_options = {
    "uri": mongo_uri,
    "database": "MVP1",
    "collection": "Projects",
    "username": "mongodbUsername",
    "password": "mongodbPassword",
    "partitioner": "MongoSamplePartitioner",
    "partitionerOptions": {
        "partitionSizeMB": "10",
        "partitionKey": "_id"
    }
}

Additionally, ensure that your Glue permission policy includes the glue:GetJobRun* action to maintain proper access and functionality.

profile picture
EXPERTE
beantwortet vor 3 Monaten
  • Hello Osvaldo,

    Thank you for your reply.

    I still get the same error.

    This is my current configuration :

    MongoDB_node1708273422672 = glueContext.create_dynamic_frame.from_options( connection_type="mongodb", connection_options={ "uri": "mongodb+srv://proj1.z1qzqz6.mongodb.net:27017/MVP1", "database": "MVP1", "collection": "Projects", "username": "mongodbUsername", "password": "mongodbPassword", "partitioner": "MongoSamplePartitioner", "partitionerOptions": { "partitionSizeMB": "10", "partitionKey": "_id" } }, transformation_ctx="MongoDB_node1708273422672", )


    My policy has : "glue:*", so I should have proper access and functionality.

    Thank you very much for your help, Marc

  • I've identified the problem, Marc. When using glueContext.create_dynamic_frame.from_options, the connection type is limited to supporting only the following sources: S3, MySQL, PostgreSQL, Redshift, SQL Server, Oracle, and DynamoDB.

    That doesn't support MongoDB

    Resources:

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen