Serverless LLMs


I've kinda multiple questions to ask

  1. Can I deploy a model serverlessly on Sagemaker? My current requirements are some pretrained models with around 8-12GB of weights and stuff.
  2. What is the best approach for LLM deployment and functioning on sagemaker? if I wanna plug some models from github and choose to manage them here on sagemaker. I might need fine-tuning thing along the way.
asked 2 months ago152 views
1 Answer


Yes, you can deploy models in serverless mode. Look at this blog post for all details:

For your own deployments, see this other blog post:



profile pictureAWS
answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions