By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

How to use Neuron SDK

0

I am new to AWS Neuron SDK and the documentation seems confusing to me. There is no direct guide on how to install the SDK and use it to compile models. The examples are outdated and the installation process is not clear.

I want to compile my models for Inferential on a CICD pipeline and deploy it. The instructions all suggest running an inf instance for compilation. I followed this to install the Inferentia on my local environment. Here is what I have done:

python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com
python -m pip install tensorflow-neuron[cc] "protobuf"
python -m pip install tensorboard-plugin-neuron

Then I looked at this example. The simpale_save has been deprecated. I could not fix it by sing the save method either.

Other examples have like this is hard to follow. I did not understand why the compilation process is so complicated in this example.

Is there any other documentations?

  • donkrets-AWS
    5 months ago

    Hi Mosi. It would help to better understand what you are trying to accomplish. You mentioned CI/CD – are you trying to continuously develop a model over time and automate its deployment for testing or automate the deployment of an existing model based on incoming inference requests, or something else?

    I suggest dividing the work into two steps. First, manually getting a Neuron environment and instance properly set up such that you can successfully run the sample apps. The existing Neuron documentation should be helpful as the majority of the Neuron documentation relates to building a specific model, and then deploying the resulting artifact (NEFF file) under a long-lived EC2 instance to handle inference traffic.

    Second, tackle the CI/CD steps needed to automate the instance setup and model deployment. The Neuron documentation currently doesn’t prescribe model deployment so this would require some design and docs beyond what Neuron offers today. Some customer find that the automation offered by Sagemaker helps with this process as Sagemaker supports inf1 instances.

Topics
Machine Learning & AI
Tags
AWS Inferentia
Language
English
Mosi
asked 5 months ago260 views
1 Answer
0

It looks like you tried to run the inf2 installation with inf1 example code. These run different versions of the tensorflow integration. Please take a look at the inf2 tutorials e.g. https://awsdocs-neuron.readthedocs-hosted.com/en/latest/src/examples/tensorflow/tensorflow-neuronx/tfneuronx-roberta-base-tutorial.html with inf2.

The correct installation procedures are listed for inf1, linked from the tutorial.

AWS
mrnikwaws
answered 4 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content