Is there any way to update Action Space when Clone a model?

Hi,

The action space and your model are closely linked - the model would have been tuned to know when best to use that particular action. The internal structure of the model also means there has to be the same number of output actions as the model was built for (or it won't have all the internal structures to give weighting to extra actions). Therefore if you alter the actions, your model will most likely be unstable.

That said, it is technically possible to alter the existing actions - but you can't add additional actions (at least not that I've discovered). To alter existing actions, you need to edit the 'model_metadata.json' file for your source model in the 'model-metadata' folder in S3. Then when you clone your source model, it will copy those new settings and create you a new model that is created. Don't forget to restore the source model back to its proper configuration.

I've had some success with marginally adjusting speeds and steering, and then retraining it for some time. But it is generally much better just to start with a fresh model, as eventually you'll over train and risk running into the NaN issue (see my other post in the forum).

Lyndon

Thanks for the detail Etaggel! After the model is first created, the number of nodes in the final layer of the neural network is fixed to match the number of actions in the action space. Thus it is not possible to increase or decrease the number of actions in the action space once the model is trained. Furthermore, changes to the action space may also not work well as the model has some "trained" expectation off the outcome of the action based on training, and this would now need to be "retrained".