I am new to AWS Neuron SDK and the documentation seems confusing to me.
There is no direct guide on how to install the SDK and use it to compile models. The examples are outdated and the installation process is not clear.
I want to compile my models for Inferential on a CICD pipeline and deploy it. The instructions all suggest running an inf instance for compilation.
I followed this to install the Inferentia on my local environment. Here is what I have done:
python -m pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com
python -m pip install tensorflow-neuron[cc] "protobuf"
python -m pip install tensorboard-plugin-neuron
Then I looked at this example. The simpale_save has been deprecated. I could not fix it by sing the save method either.
Other examples have like this is hard to follow. I did not understand why the compilation process is so complicated in this example.
Is there any other documentations?
Hi Mosi. It would help to better understand what you are trying to accomplish. You mentioned CI/CD – are you trying to continuously develop a model over time and automate its deployment for testing or automate the deployment of an existing model based on incoming inference requests, or something else?
I suggest dividing the work into two steps. First, manually getting a Neuron environment and instance properly set up such that you can successfully run the sample apps. The existing Neuron documentation should be helpful as the majority of the Neuron documentation relates to building a specific model, and then deploying the resulting artifact (NEFF file) under a long-lived EC2 instance to handle inference traffic.
Second, tackle the CI/CD steps needed to automate the instance setup and model deployment. The Neuron documentation currently doesn’t prescribe model deployment so this would require some design and docs beyond what Neuron offers today. Some customer find that the automation offered by Sagemaker helps with this process as Sagemaker supports inf1 instances.