Selenium and Web-Drivers in lambda function

0

Hello everyone, Our team is working on a web scraping project, that we'd like to upload to aws lambda. But, unfortunately, we've encountered some problems with web drivers. Based on our research, we are using: Python 3.8 Selenium v4.15.1 Chrome driver v121.0.6167.139 Headless-Chromium v1.0.0-57 (web drivers are packed together, and added to a layer through S3) as described in: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Synthetics_Library_python_selenium.html We've modeled our code according to those few tutorials we were able to find (such as https://www.youtube.com/watch?v=b49Y3NGJX68). But while trying to run even the simplest code, we encounter one of two errors: "Service /opt/chromedriver unexpectedly exited. Status code was: 127", or "Unable to locate or obtain driver for chrome".

We are using the following code structure: def lambda_handler(event, context):

logging.info('Web driver started')

# Define path
service = Service(executable_path=r'/opt/chromedriver')

# Define options
chrome_options = Options()

chrome_options.binary_location = '/opt/headless-chromium'

chrome_options.add_argument("--headless")

# Initialize Chrome
driver = webdriver.Chrome(service=service, options=chrome_options)
logging.info('Web driver defined')

# Open Google and then quit the driver
driver.get('https://www.google.ru')
logging.info('Web driver opened')

driver.quit()
logging.info('Web driver closed')

return "finished"

We'd be truly thankful if anyone could help us understand the reasons behind the mentioned errors.

  • Are the chromedriver and the headless-chromium in your Lambda layer built for the correct CPU architecture of your Lambda function? (i.e arm64 vs x86_64)

profile picture
Giorgi
질문됨 한 달 전110회 조회
답변 없음

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인