error while reading csv from S3 using pandas

0

Hi Team, I am getting an error "Install s3fs to access S3" while trying to access a file from S3 in AWS lambda. However I tried importing all possible packages but still I am not able to read the file.

  • import json import boto3 import pandas as pd

    def lambda_handler(event, context):

    df1 = pd.read_csv('s3://himanrcctest/size_and_fit/Logs_2023-07-15.csv001',delimiter="\t")
    return str(df1
    

    I have already run below commands: pip3 install s3fs pip3 install fs_s3fs however I got this error while running above commnd - awscli 1.29.60 requires botocore==1.31.60, but you have botocore 1.31.17 which is incompatible. boto3 1.28.60 requires botocore<1.32.0,>=1.31.60, but you have botocore 1.31.17 which is incompatible.

질문됨 7달 전328회 조회
1개 답변
0

Hello.

What if you run the command below to update botocore to the latest version?
https://docs.aws.amazon.com/efs/latest/ug/install-botocore.html

sudo pip3 install botocore --upgrade

Also, since neither pandas nor s3fs are included in the Lambda package, you need to create a layer using the steps in the document below.

mkdir python
pip3 install pandas -t ./python/
pip3 install s3fs -t ./python/
zip -r lambda_layer.zip python/
aws lambda publish-layer-version --layer-name "lambda_layer_name" --zip-file fileb://lambda_layer.zip --compatible-runtimes python3.11
profile picture
전문가
답변함 7달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠