Questions tagged with AWS Inferentia
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
We have a huggingfacemodel with zero-shot-classification with neuron infernetia. It's based on [the pretrained huggingface pipelines distilBert with TensorFlow2...
1
answers
0
votes
331
views
asked a year agolg...
Hello,
Difference between AWS Trainium, AWS inferentia and Instances with Habana Accelerators?
Thanks,
1
answers
0
votes
418
views
asked a year agolg...
I converted a pytorch BERT model to neuron. However the embedding or output tensors which is a list of 1024 size is different..i.e the list sizes are same but individual entries differ. Each of the...
2
answers
0
votes
530
views
asked 2 years agolg...
Hello,
We are testing the pipeline mode for neuron/inferentia, but can not get a model running for multi-core. The single core compiled model loads fine and is able to run inference on inferentia...
1
answers
0
votes
442
views
asked 2 years agolg...
Im using the following code to load a neuron compiled model for inference. However on my inf1.2xlarge instance, neuron-top shows for cores (NC0 to NC3). Only NC0 gets used in inference. How do I...
1
answers
0
votes
325
views
asked 2 years agolg...
Hi,
I want to neuron compile a bert large model(patentbert from google) which has sequence length 512. How do I do this?
Also I want to call the model as before or need to know what I should change...
1
answers
0
votes
426
views
asked 2 years agolg...
I am trying to load a neuron compiled model generated as given in https://awsdocs-neuron.readthedocs-hosted.com/en/latest/src/examples/tensorflow/huggingface_bert/huggingface_bert.html . I am still a...
2
answers
0
votes
396
views
asked 2 years agolg...
Hi,
This link https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/tensorflow/tensorflow-neuron/tutorials/bert_demo/bert_demo.html mentions how to compile using tensorflow 1. Can anyone...
1
answers
0
votes
465
views
asked 2 years agolg...
I followed user guide on updating torch neuron and then started compiling the model to neuron.
But got an error, from which I don't understand what's wrong.
In Neuron SDK you claim that it should...
1
answers
1
votes
820
views
asked 2 years agolg...
I'm following some guides and from my understanding this should be possible. But I've been trying for hours to compile a yolov5 model into a neuron model with no success. Is it even possible to do...
1
answers
2
votes
720
views
asked 2 years agolg...
Hi Team,
I wanted to compile a BERT model and run it on inferentia. I trained my model using pytorch and tried to convert it by following the same steps in this...
1
answers
0
votes
356
views
asked 2 years agolg...
I am trying to test a model compiled for Inferentia on an `inf1.2xlarge`, but when loading the model I receive the following error messages:
```
2022-Sep-15 22:10:01.0152 3802:3802 ERROR ...
1
answers
0
votes
347
views
asked 2 years agolg...