AWS Lambda Timing out after minutes to simply get a CSV file from a bucket.

0

I have a csv file in my s3 bucket called delte.csv located in the following URI: s3://aerocastwx.com/model_csv/delte.csv

My lambda function I know imports pandas and boto (it works when I just have the import statements) All I want for now is the function to load the csv as a dataframe. I know in this instance the dataframe has nothing to do with the output. The following is my lambda code: **EDIT: The code stalls once I add this line **

response = s3.get_object(Bucket=bucket_name, Key=file_name)

**That is the line that is causing it to stall out. **

import json
import pandas
import datetime
import boto3

#import cartopy

def lambda_handler(event, context):
    x = event['queryStringParameters']['x']
    y = event['queryStringParameters']['y']

    print(f" x:{x} , y:{y}")

    res_body = {}
    res_body['x'] = str(x)
    res_body['y'] = str(y)
    res_body['ans'] = add(x,y)

    http_res = {}
    http_res['statusCode'] = 200
    http_res['headers'] = {
        "Content-Type": "application/json",
        "Access-Control-Allow-Headers": "Content-Type",
        "Access-Control-Allow-Origin": "*",
        "Access-Control-Allow-Methods": "GET"
    }
    http_res['body'] = json.dumps(res_body)

    return http_res
def add(x,y):
    bucket_name = "aerocastwx.com"
    file_name = "model_csv/delte.csv"
    print("Getting S3")
    # Create an S3 client
    s3 = boto3.client('s3')

    try:
        # Get the CSV file from S3
        response = s3.get_object(Bucket=bucket_name, Key=file_name)

        # Read the CSV data into a Pandas DataFrame
        df = pd.read_csv(response['Body'])

        # Now you can work with the DataFrame (e.g., perform data analysis)
        # For example, you can print the first few rows of the DataFrame
        print(df.head())

        return {
            'statusCode': 200,
            'body': 'CSV file loaded successfully.'
        }

    except Exception as e:
        return {
            'statusCode': 500,
            'body': str(e)
        }
    return x+y+y

Below this is my basic settings: Basic settings for Lambda Function

Below is my role permissions: Permission for IAM user Enter image description here

Possible VPC Endpoint Issure: Based on what I have seen it may be a VPC end point issue. I am really bad at VPC's so I think this is the problem. This is what I have now... Connected to the same VPC as my Lambda function (I need this vpc to be enabled).

Enter image description here

Enter image description here

Route Table: Enter image description here Enter image description here

Kevin
asked 8 months ago299 views
2 Answers
0

There are two ways for lambda function within VPC to connect to s3:

  1. Either set up an S3 VPC endpoint or NAT Gateway. Refer this AWS Documentation for same.

  2. Or If you prefer not to go to public network from Lambda, use S3 VPC endpoint. Refer this AWS Documentation for same.

Hope this helps.

Comment here if you have additional questions, happy to help.

Abhishek

profile pictureAWS
EXPERT
answered 8 months ago
  • Yes I tried the S3 VPC end point. Look at the images above

  • Share snapshot of your route table, does route table has endpoint route to it?

  • Just updated the post with snippits of route table.

  • Route table id and VPC Endpoint end differ from what you mentioned first and just now. First VPCE ends with "22a8" and second one is ending with "6476" and same for route table too. I'd suggest you to follow the steps exactly as mentioned at Create S3 VPC gateway endpoint.

  • I just redid it and uploaded the updated VPE. They should be the same. Still does not work.

0

Create yourself an S3 gateway endpoint so that your lambda function can route to the S3 service.

From the screen shot it looks like you’ve not used the gateway.

profile picture
EXPERT
answered 8 months ago
  • HI Gary, Thank you for answering! I tried changing it and still does not work.. I updated the photos in my post with my newest endpoint

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions