How to resolve connection failures from Glue to Opensearch?

Question

Trying to write to my OpenSearch cluster

"An error occurred while calling o265.pyWriteDynamicFrame. Connection error (check network and/or proxy settings)- all nodes failed; tried [[https://---.us-west-2.es.amazonaws.com:443]]"

Seemed to work with less data. I've set everything up according to the instructions.

ElasticsearchConnector7134forAWSGlue30_node1688409952934 = glueContext.write_dynamic_frame.from_options(
    frame=DropandRename_node1688409839967,
    connection_type="marketplace.spark",
    connection_options={
        "path": "path-to-my-index",
        "es.nodes.wan.only": "true",
        "es.nodes": "https://---.us-west-2.es.amazonaws.com",
        "es.port": "443",
        "connectionName": "connection",
    },
    transformation_ctx="ElasticsearchConnector7134forAWSGlue30_node1688409952934",
)

Answer

Notice that error doesn't mean there is a connectivity issue (especially if you say it works with less data), it means it failed writing and that can be for a number of reasons, for instance the server nodes running out of disk space. 
Check the lines around that error in the log for some further indication on the issue, also you should check the server health status.

How to resolve connection failures from Glue to Opensearch?

Relevanter Inhalt