Loading streaming data from dynamodb stream to ElasticSearch via lambda is described here. I am facing following errors when triggering lambda for test event.
{"error":"no handler found for uri [//lambda-index/_doc/CUSTOMER] and method [PUT]"}
{"error":"no handler found for uri [//lambda-index/_doc/CUSTOMER] and method [DELETE]"}
Checked the same api calls at OpenSearch Dashboard Console and apis work fine:
PUT /lambda-index/_doc/CUSTOMER#34567
DELETE /lambda-index/_doc/CUSTOMER#34567
This suggests me that the error is somehow in formatting URL while making call via request() module in Python. Also the character '#' (inside CUSTOMER#34567) in the api call gets truncated (CUSTOMER#12345 and CUSTOMER#987654 are both taken up as CUSTOMER when they should be different and as given).
Do you have an idea to correct the mistake here.
Lambda code:
import boto3
import requests
from requests_aws4auth import AWS4Auth
region = 'ap-south-1' # e.g. us-east-1
service = 'es'
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)
host = 'https://vpc-elasticsearch-temp-au5shucd5nkecivdx7cnm7jaqi.ap-south-1.es.amazonaws.com/'
index = 'lambda-index'
type = '_doc'
url = host + '/' + index + '/' + type + '/'
headers = { "Content-Type": "application/json" }
def handler(event, context):
count = 0
for record in event['Records']:
# Get the primary key for use as the OpenSearch ID
id = record['dynamodb']['Keys']['PK']['S']
if record['eventName'] == 'REMOVE':
r = requests.delete(url + id, auth=awsauth)
else:
document = record['dynamodb']['NewImage']
r = requests.put(url + id, auth=awsauth, json=document, headers=headers)
count += 1
return str(count) + ' records processed.'
Versions:
Python 3.9
requests v 2.27.1
ElasticSearch: OpenSearch1.2
Omg thank you!!