Questions tagged with AWS Inferentia
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
Is there more documentation/examples for *TensorFlow* on Trn1/Trn1n instances?
Documentation at:
[https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/tensorflow/index.html]()...
2
answers
0
votes
286
views
asked 24 days agolg...
We are using tensorflow.neuron to compile a tensorflow 1.x SavedModel to run on AWS Inferentia machines on EC2. We do this by calling:
tensorflow.neuron.saved_model.compile(model_dir,...
3
answers
0
votes
404
views
asked 2 months agolg...
Currently, I host my model with `tensorflow_model_server`. Here is how I export my model:
```
model = tf.keras.models.load_model("model.hdf5")
def __decode_images(images, nch):
o =...
1
answers
0
votes
458
views
asked 6 months agolg...
I am new to AWS Neuron SDK and the documentation seems confusing to me.
There is no direct guide on how to install the SDK and use it to compile models. The examples are outdated and the installation...
1
answers
0
votes
571
views
asked 6 months agolg...
Currently, we are using Elastic Inference for inferencing on AWS ECS. We use `inference_accelerators` in `ecs.Ec2TaskDefinition` to set up elastic inference. For scaling, we are monitoring...
1
answers
0
votes
517
views
asked 6 months agolg...
I have a project where I would like to send inference requests. For this I need a API as AWS Lambda or a SageMaker endpoint so that the customer can send their request there.
The inference performed...
1
answers
0
votes
672
views
asked 7 months agolg...
Hello, i'm using **pytorch-inference:2.0.1-gpu-py310-cu118-ubuntu20.04-sagemaker** AMI and following this guide...
1
answers
0
votes
490
views
asked 7 months agolg...
hello,
i'm using **pytorch-inference:2.0.1-gpu-py310-cu118-ubuntu20.04-sagemaker** image to run the service i'd like to switch to INF2 instance.
~~I think i can try to use...
1
answers
0
votes
536
views
asked 8 months agolg...
TensorFlow instancelg...
I'm considering launching an instance to work on one of my TensorFlow models since my current PC doesn't perform efficiently. My PC has 32GB of RAM, a 20CPU i7 processor, and an RTX 3050Ti 20GB GPU. I...
1
answers
0
votes
522
views
asked 8 months agolg...
Hello,
I am using an autoscaling group with inferentia chips but I encounter some problems during the deployment. There are three available zones in my ASG which means that those zones must contain...
0
answers
0
votes
393
views
asked 8 months agolg...
As the title says, we can host LLM's and Stable diffusion models from jumpstart directly on SageMaker Inf1 or Inf2 chips ?
> I tried doing that with Stable Diffusion 2 Model (i.e from studio...
2
answers
0
votes
722
views
asked 10 months agolg...
We are facing issues while using this model on the aforementioned machine. We were able to run the same experiment on G5 instance successfully but we are observing that the same code is not working on...
1
answers
0
votes
542
views
asked 10 months agolg...