Amazon Neptune Serverless LOAD_UNEXPECTED_ERROR

0

Has anyone experienced an issue like this when executing a load job on a Neptune serverless instance configured to scale from 1 to 32 NCUs? This load seems to have failed with an unexpected error, but all of the error indicators in the status response below show that there were no errors. It also seems like the load never actually started as the only entry in the feedCount element is LOAD_NOT_STARTED.

I had previously loaded a few billion vertices successfully, but am having trouble loading the edges. I am attempting to load all OpenCypher formatted .gz files in a given S3 prefix. If I change the launch request to specify the first file in the prefix, it does load fine and complete without error. However, I get the status response below after about 7.5 hours when attempting to load all the files at once.

Is there any way for me to get more details on what the possible failure might be?

{
    "status" : "200 OK",
    "payload" : {
        "feedCount" : [
            {
                "LOAD_NOT_STARTED" : 3171
            }
        ],
        "overallStatus" : {
            "fullUri" : "s3://my-bucket/neptune/data/edge",
            "runNumber" : 1,
            "retryNumber" : 1,
            "status" : "LOAD_UNEXPECTED_ERROR",
            "totalTimeSpent" : 26754,
            "startTime" : 1692320967,
            "totalRecords" : 0,
            "totalDuplicates" : 0,
            "parsingErrors" : 0,
            "datatypeMismatchErrors" : 0,
            "insertErrors" : 0
        },
        "errors" : {
            "startIndex" : 0,
            "endIndex" : 0,
            "loadId" : "637f253f-f0ef-419e-a8b6-599eceea2222",
            "errorLogs" : [ ]
        }
    }
}

I launched the load job with the following command:

curl -X POST https://neptune-dev-1.cluster-cwcwpnhm4dly.us-east-1.neptune.amazonaws.com:8182/loader \
  -H 'Content-Type: application/json' \
  -d '{
        "source" : "s3://my-bucket/neptune/data/edge",
        "iamRoleArn" : "arn:aws:iam::<accountId>:role/NeptuneLoadFromS3",
        "format" : "opencypher",
        "region" : "us-east-1",
        "failOnError" : "TRUE",
        "parallelism" : "OVERSUBSCRIBE",
        "userProvidedEdgeIds": "FALSE"
      }'

Thanks for reading! Paul E.

질문됨 10달 전241회 조회
2개 답변
0

You can view the details of errors and the full error log by appending ?details=true&errors=true to the end of the /loader status request. As documented here: https://docs.aws.amazon.com/neptune/latest/userguide/load-api-reference-error-logs-examples.html

profile pictureAWS
답변함 10달 전
  • Thanks Taylor! That's the really confounding thing. The response I posted in my initial question was generated with those parameters. :(

  • If you're submitting that via curl or awscurl, be sure to enclose the entire URL in double quotes. I've seen situations where everything after the first & will get ignored otherwise.

0

Hi, This post may help you locate and resolve your issue with the load job: https://repost.aws/knowledge-center/neptune-bulk-loader-processing-errors

Additionally, if not too painful for you (automation possible ?), I'd suggest to try to load all your files one by one to see on which you encounter the error(s).

Hope it helps!

Didier

profile pictureAWS
전문가
답변함 10달 전
  • Thanks Didier! I had seen that article, but since the status api doesn't show any errors, I wasn't able to make any progress with it. I think the idea of automating the job submission to submit one file at a time is the way I'll try to figure this out. Thanks for the suggestion!!

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠