I have a lambda and a neptune DB in the same VPC. I am trying to read from the neptune DB stream, but when running the lambda I am getting a connection error.
EndpointConnectionError: Could not connect to the endpoint URL: \"https://neptuneclusteridinstance123-asdf123.abcd.us-east-1.neptune.amazonaws.com:8182/propertygraph/stream?limit=100&iteratorType=AT_SEQUENCE_NUMBER&commitNum=0\
Any ideas on what is going wrong?
CDK:
const vpc = new ec2.Vpc(this, "vpc", {
ipAddresses: ec2.IpAddresses.cidr('10.0.0.0/20'),
natGateways: 1,
maxAzs: 2,
subnetConfiguration: [
{
name: 'private-subnet-1',
subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
cidrMask: 26
},
{
name: 'public-subnet-1',
subnetType: ec2.SubnetType.PUBLIC,
cidrMask: 26
}
]
})
// Get lists of Subnets by type
var privateSubnets = vpc.privateSubnets;
// Create Subnet group list to be used with Neptune.
const neptuneSubnets: ec2.SubnetSelection = { subnets: privateSubnets };
const subnetGroup = new neptune.SubnetGroup(this, 'subnet_group', {
vpc: vpc,
vpcSubnets: {
subnets: privateSubnets
}
})
// Neptune cluster parameter group
const neptuneClusterParameterGroup = new neptune.ClusterParameterGroup(this, 'neptuneClusterParameterGroup', {
description: "the main purpose of this parameter group is to enable neptune streams",
family: neptune.ParameterGroupFamily.NEPTUNE_1_3,
parameters: {
neptune_streams: '1'
}
})
const neptune_cluster = new neptune.DatabaseCluster(this, 'neptune_cluster_id', {
vpc: vpc,
vpcSubnets: neptuneSubnets,
subnetGroup: subnetGroup,
instanceType: neptune.InstanceType.T3_MEDIUM,
deletionProtection: false,
clusterParameterGroup: neptuneClusterParameterGroup
})
// lambda
const neptuneLambda = new lambda.Function(this, "neptuneLambda", {
vpc: vpc,
vpcSubnets: {
subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS
},
memorySize: 512, // MB
runtime: lambda.Runtime.PYTHON_3_8,
code: lambda.Code.fromAsset(path.join(__dirname, '../src/lambda')),
handler: 'neptuneLambdaHandler.handler',
timeout: cdk.Duration.seconds(120)
})
Lambda:
neptuneData = boto3.client(
'neptunedata',
endpoint_url=neptune_endpoint_URL)
response = neptuneData.get_propertygraph_stream(
limit=100,
iteratorType='AT_SEQUENCE_NUMBER',
commitNum=int(startingCommitNum)
)
You'll likely need two security groups here. The first one you'll want to assign to your Lambda function. The second, you'll want to assign to your Neptune cluster/instances. Within the second security group, you'll want to allow traffic FROM (incoming) the Lambda function security group ID on port 8182.