Amazon Neptune Serverless LOAD_UNEXPECTED_ERROR

0

Has anyone experienced an issue like this when executing a load job on a Neptune serverless instance configured to scale from 1 to 32 NCUs? This load seems to have failed with an unexpected error, but all of the error indicators in the status response below show that there were no errors. It also seems like the load never actually started as the only entry in the feedCount element is LOAD_NOT_STARTED.

I had previously loaded a few billion vertices successfully, but am having trouble loading the edges. I am attempting to load all OpenCypher formatted .gz files in a given S3 prefix. If I change the launch request to specify the first file in the prefix, it does load fine and complete without error. However, I get the status response below after about 7.5 hours when attempting to load all the files at once.

Is there any way for me to get more details on what the possible failure might be?

{
    "status" : "200 OK",
    "payload" : {
        "feedCount" : [
            {
                "LOAD_NOT_STARTED" : 3171
            }
        ],
        "overallStatus" : {
            "fullUri" : "s3://my-bucket/neptune/data/edge",
            "runNumber" : 1,
            "retryNumber" : 1,
            "status" : "LOAD_UNEXPECTED_ERROR",
            "totalTimeSpent" : 26754,
            "startTime" : 1692320967,
            "totalRecords" : 0,
            "totalDuplicates" : 0,
            "parsingErrors" : 0,
            "datatypeMismatchErrors" : 0,
            "insertErrors" : 0
        },
        "errors" : {
            "startIndex" : 0,
            "endIndex" : 0,
            "loadId" : "637f253f-f0ef-419e-a8b6-599eceea2222",
            "errorLogs" : [ ]
        }
    }
}

I launched the load job with the following command:

curl -X POST https://neptune-dev-1.cluster-cwcwpnhm4dly.us-east-1.neptune.amazonaws.com:8182/loader \
  -H 'Content-Type: application/json' \
  -d '{
        "source" : "s3://my-bucket/neptune/data/edge",
        "iamRoleArn" : "arn:aws:iam::<accountId>:role/NeptuneLoadFromS3",
        "format" : "opencypher",
        "region" : "us-east-1",
        "failOnError" : "TRUE",
        "parallelism" : "OVERSUBSCRIBE",
        "userProvidedEdgeIds": "FALSE"
      }'

Thanks for reading! Paul E.

已提問 10 個月前檢視次數 241 次
2 個答案
0

You can view the details of errors and the full error log by appending ?details=true&errors=true to the end of the /loader status request. As documented here: https://docs.aws.amazon.com/neptune/latest/userguide/load-api-reference-error-logs-examples.html

profile pictureAWS
已回答 10 個月前
  • Thanks Taylor! That's the really confounding thing. The response I posted in my initial question was generated with those parameters. :(

  • If you're submitting that via curl or awscurl, be sure to enclose the entire URL in double quotes. I've seen situations where everything after the first & will get ignored otherwise.

0

Hi, This post may help you locate and resolve your issue with the load job: https://repost.aws/knowledge-center/neptune-bulk-loader-processing-errors

Additionally, if not too painful for you (automation possible ?), I'd suggest to try to load all your files one by one to see on which you encounter the error(s).

Hope it helps!

Didier

profile pictureAWS
專家
已回答 10 個月前
  • Thanks Didier! I had seen that article, but since the status api doesn't show any errors, I wasn't able to make any progress with it. I think the idea of automating the job submission to submit one file at a time is the way I'll try to figure this out. Thanks for the suggestion!!

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南