how to fine tune an existing model in sagemaker?

Question

I am planning to fine tune a BERT based model (DistilBERT base model (uncased). the fine tuning process, as i understand , would involve loading the weights for the DistilBERT model. examples i have seen use hugging face library to download model and fine tune it, but can i do this in sagemaker with pytorch library without using hugging face?

Answer

You can fine tune BERT models on SageMaker. Although huggingface provides an easy integration with their Huggingface estimator something like this. You are free to use sagemaker pytorch estimators to fine tune models. You can refer the below example for fine tuning bert using pytorch estimator.

https://github.com/aws/amazon-sagemaker-examples/blob/main/training/smart_sifting/Text_Classification_BERT/Train_text_classification.ipynb

how to fine tune an existing model in sagemaker?

Relevant content