Question about Fine-Tuning Model for Dialogue
First of all, great work!
I'm super interested in extending this checkpoint for more interactive ways of using it, and hence the following question:
I was wondering if I could get some pointers on how to fine-tune this model to create a chatbot checkpoint, e.g., user asks how to do BFS in Python, and model generates BFS in Python or vice versa. Is there a particular dataset you would recommend to try out?
Thanks so much!
+1. Same question.
You can check this blog about StarChat an instruction-tuned version of StarCoder, the fine-tuning code is available at bigcode/starcoder along with a demo: http://hf.co/spaces/HuggingFaceH4/starchat-playground.
@loubnabnl the fine-tuned StarCoderBase model on huggingchat and the starcoder are not the same though. The first one is fine-tuned based on StarCoderBase, while the other is fine-tuned based on dolly.
The StarCoderBase on the Hugging Chat is not fine-tuned is was just prompted with a series of dialogue. StarChat is a fine-tuned version of StarCoderBase on the OpenAssistant and Dolly datasets
@loubnabnl Gotcha. Thank you for the explanation!
Thank you!