Lambda (inside VPC) using Neptune API results in error: EndpointConnectionError: Could not connect to the endpoint URL

0

I have a lambda and a neptune DB in the same VPC. I am trying to read from the neptune DB stream, but when running the lambda I am getting a connection error.

EndpointConnectionError: Could not connect to the endpoint URL: \"https://neptuneclusteridinstance123-asdf123.abcd.us-east-1.neptune.amazonaws.com:8182/propertygraph/stream?limit=100&iteratorType=AT_SEQUENCE_NUMBER&commitNum=0\

Any ideas on what is going wrong?

CDK:


const vpc = new ec2.Vpc(this, "vpc", {
      ipAddresses: ec2.IpAddresses.cidr('10.0.0.0/20'),
      natGateways: 1,
      maxAzs: 2,
      subnetConfiguration: [
        {
          name: 'private-subnet-1',
          subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
          cidrMask: 26
        },
        {
          name: 'public-subnet-1',
          subnetType: ec2.SubnetType.PUBLIC,
          cidrMask: 26
        }
      ]
    })

    // Get lists of Subnets by type
    var privateSubnets = vpc.privateSubnets;

    // Create Subnet group list to be used with Neptune.
    const neptuneSubnets: ec2.SubnetSelection = { subnets: privateSubnets };

    const subnetGroup = new neptune.SubnetGroup(this, 'subnet_group', {
      vpc: vpc,
      vpcSubnets: {
        subnets: privateSubnets
      }
    })

    // Neptune cluster parameter group
    const neptuneClusterParameterGroup = new neptune.ClusterParameterGroup(this, 'neptuneClusterParameterGroup', {
      description: "the main purpose of this parameter group is to enable neptune streams",
      family: neptune.ParameterGroupFamily.NEPTUNE_1_3,
      parameters: {
        neptune_streams: '1'
      }
    })

    const neptune_cluster = new neptune.DatabaseCluster(this, 'neptune_cluster_id', {
      vpc: vpc,
      vpcSubnets: neptuneSubnets,
      subnetGroup: subnetGroup,
      instanceType: neptune.InstanceType.T3_MEDIUM,
      deletionProtection: false,
      clusterParameterGroup: neptuneClusterParameterGroup
    })

    // lambda
    const neptuneLambda = new lambda.Function(this, "neptuneLambda", {
      vpc: vpc,
      vpcSubnets: {
        subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS
      },
      memorySize: 512, // MB
      runtime: lambda.Runtime.PYTHON_3_8,
      code: lambda.Code.fromAsset(path.join(__dirname, '../src/lambda')),
      handler: 'neptuneLambdaHandler.handler',
      timeout: cdk.Duration.seconds(120) 
    })

Lambda:

neptuneData = boto3.client(
    'neptunedata',
    endpoint_url=neptune_endpoint_URL)

response = neptuneData.get_propertygraph_stream(
        limit=100,
        iteratorType='AT_SEQUENCE_NUMBER',
        commitNum=int(startingCommitNum)
    )
1 Answer
1

Hello.

As far as I can see from the error message, Lambda cannot access the Neptune cluster endpoint.
Also, when looking at the CDK code, there is no security group defined, so I think that port 8182 is not allowed in the Neptune cluster's security group, causing an error.
So, first check the security group settings.
https://docs.aws.amazon.com/neptune/latest/userguide/get-started-vpc.html#security-vpc-security-group

profile picture
EXPERT
answered a month ago
profile picture
EXPERT
reviewed a month ago
profile pictureAWS
EXPERT
reviewed a month ago
  • You'll likely need two security groups here. The first one you'll want to assign to your Lambda function. The second, you'll want to assign to your Neptune cluster/instances. Within the second security group, you'll want to allow traffic FROM (incoming) the Lambda function security group ID on port 8182.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions