1 Resposta
- Mais recentes
- Mais votos
- Mais comentários
0
It looks like you tried to run the inf2 installation with inf1 example code. These run different versions of the tensorflow integration. Please take a look at the inf2 tutorials e.g. https://awsdocs-neuron.readthedocs-hosted.com/en/latest/src/examples/tensorflow/tensorflow-neuronx/tfneuronx-roberta-base-tutorial.html with inf2.
The correct installation procedures are listed for inf1, linked from the tutorial.
respondido há 5 meses
Conteúdo relevante
- AWS OFICIALAtualizada há 4 meses
- AWS OFICIALAtualizada há 2 anos
Hi Mosi. It would help to better understand what you are trying to accomplish. You mentioned CI/CD – are you trying to continuously develop a model over time and automate its deployment for testing or automate the deployment of an existing model based on incoming inference requests, or something else?
I suggest dividing the work into two steps. First, manually getting a Neuron environment and instance properly set up such that you can successfully run the sample apps. The existing Neuron documentation should be helpful as the majority of the Neuron documentation relates to building a specific model, and then deploying the resulting artifact (NEFF file) under a long-lived EC2 instance to handle inference traffic.
Second, tackle the CI/CD steps needed to automate the instance setup and model deployment. The Neuron documentation currently doesn’t prescribe model deployment so this would require some design and docs beyond what Neuron offers today. Some customer find that the automation offered by Sagemaker helps with this process as Sagemaker supports inf1 instances.