Skip to content

All Content tagged with Amazon SageMaker Deployment

Amazon SageMaker provides a broad selection of machine learning (ML) infrastructure and model deployment options to help meet your needs, whether real time or batch. Once you deploy a model, SageMaker creates persistent endpoints to integrate into your applications to make ML predictions (also known as inference). It supports the entire spectrum of inference, from low latency (a few milliseconds) and high throughput (hundreds of thousands of inference requests per second) to long-running inference for use cases such as natural language processing (NLP) and computer vision (CV). Whether you bring your own models and containers or use those provided by AWS, you can implement MLOps best practices using SageMaker to reduce the operational burden of managing ML models at scale.

Content language: English

Filter content
Select tags to filter
Sort by
Sort by most recent
96 results
We are experiencing recurring InsufficientInstanceCapacity errors with a SageMaker real-time inference endpoint in ca-central-1 (Canada Central). Our workload is an LLM inference endpoint that requir...
4
answers
0
votes
174
views

asked 2 months ago

**Context** * Base image (AWS DLC): 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.6.0-transformers4.49.0-gpu-py312-cu124-ubuntu22.04 * This had been stable for months. ...
1
answers
0
votes
173
views

asked 8 months ago

I want to use a SageMaker Inference Toolkit to create a inference docker image. I want to use this inference docker image to deploy a SageMaker endpoint using Bring Your Own Container (BYOC).
I have deployed a model in AWS Sagemaker to generate embeddings. I used huggingface-tei image with version 1.8.2. It deployed well. Initially, It works fine and after sometimes it returns None in plac...
1
answers
0
votes
132
views

asked 9 months ago

This article helps you understand and resolve the SageMaker AI endpoint deployment error that occurs when the requested instance type isn’t available in enough Availability Zones overlapping with your...
Hey everyone, I am facing some issue in deploying my endpoint on Sagemaker (later to be invoked by a downstream service - ECS). Here's my code in deploying it: ``` import os, tarfile, uuid, boto3, s...
1
answers
0
votes
249
views

asked a year ago

Below is my docker file. I can run everything locally serving bentoml just fine however in cloud watch the Sagemaker endpoint never actually initializes it loops. I have tried multiple configurations ...
2
answers
0
votes
141
views

asked a year ago

I have created a serverless endpoint for inference using my own ECR image. However, creation fails with no CloudWatch logs... I do everything I can ...
1
answers
0
votes
233
views

asked 2 years ago

![Enter image description here](/media/postImages/original/IMiXyVYzfRQp6i7mAmRVTbfQ) The figure above illustrates the directory tree for model_data model.tar.gz, where inference.py has dependency on...
2
answers
0
votes
400
views

asked 2 years ago

Hi , I have few queries regarding sagemaker. Are there any APIs that will provide- 1. DETAILED info on the running models (instances) deployed on Sagemaker endpoint ? 2. DETAILED info on the list of M...
1
answers
0
votes
355
views

asked 2 years ago

I followed AWS tutorials to deploy a model on Sagemaker, but when the server deployed takes several minutes to process a request, it runs the same processing multiple times for an invocation. I report...
2
answers
0
votes
442
views

asked 2 years ago

Hello, I am trying to serve a model using SageMaker Endpoint. I am using Triton Inference Server as a framework, I know that I can enable Triton's gRPC protocol communication by setting the `SAGEMA...
1
answers
0
votes
803
views

asked 2 years ago

  • 1
  • 2
  • 3
  • 4
  • 5
  • •••
  • 8
  • Page size
    12 / page