sagemaker notebook instance lifecycle configuration - pip install

0

I am running the following pip install, every time I launch my notebook instance.

!pip install torch
!pip install transformers

How do I add it to an existing life cycle configuration? Existing lifecycle configuration looks something like this:

#!/bin/bash

set -ex

# OVERVIEW
# This script stops a SageMaker notebook once it's idle for more than 1 hour (default time)
# You can change the idle time for stop using the environment variable below.
# If you want the notebook the stop only if no browsers are open, remove the --ignore-connections flag
#
# Note that this script will fail if either condition is not met
#   1. Ensure the Notebook Instance has internet connectivity to fetch the example config
#   2. Ensure the Notebook Instance execution role permissions to SageMaker:StopNotebookInstance to stop the notebook 
#       and SageMaker:DescribeNotebookInstance to describe the notebook.
#

# PARAMETERS
IDLE_TIME=3600

echo "Fetching the autostop script"
wget https://raw.githubusercontent.com/aws-samples/amazon-sagemaker-notebook-instance-lifecycle-config-samples/master/scripts/auto-stop-idle/autostop.py


echo "Detecting Python install with boto3 install"

# Find which install has boto3 and use that to run the cron command. So will use default when available
# Redirect stderr as it is unneeded
CONDA_PYTHON_DIR=$(source /home/ec2-user/anaconda3/bin/activate /home/ec2-user/anaconda3/envs/JupyterSystemEnv && which python)
if $CONDA_PYTHON_DIR -c "import boto3" 2>/dev/null; then
    PYTHON_DIR=$CONDA_PYTHON_DIR
elif /usr/bin/python -c "import boto3" 2>/dev/null; then
    PYTHON_DIR='/usr/bin/

I found this link: https://github.com/aws-samples/amazon-sagemaker-notebook-instance-lifecycle-config-samples/blob/master/scripts/install-pip-package-all-environments/on-start.sh

however, I'd like to install multiple packages.

asked 4 months ago197 views
1 Answer
0

Hi,

Yes, the script that you found is the way to go.

If you replace pip install --upgrade "$PACKAGE" with 2 lines of your own packages torch and transformers, it should be fine.

Best,

Didier

profile pictureAWS
EXPERT
answered 4 months ago
  • @Didier_Durand Right, I have the auto-shut lifecycle configuration in place already though. So, I am trying to add/modify the existing script. I have added the script in the op. can you please share how / where I'd add the pip install .... for multiple package installations. Do I need a pip install for all packages? Is there a way to run requirements.txt on start?

  • Hi @Didier_Durand did you have any thoughts on this? how do I add this to the existing timeout lifecycle configuration?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions