Fix the special token mapping
Have you tested this change and confirmed a positive benefit? If so can you give details of that?
I kept special_tokens_map.json the same as it was provided by the Stable Vicuna team. I did test changing it like you have here, but noticed no benefit. It still required a very specific prompt template.
If there's some known benefit to changing it then I'll happily do it. But otherwise I'd prefer to keep using the same file as provided by the original creators, as that's what they trained with.
Just noticed it from the Reddit post here: https://www.reddit.com/r/LocalLLaMA/comments/13h3tac/gptqforllama_repo_default_tokenization_issue_and/, thought it may be related. From my local test, I didn't see any significant improvement for the model output. However, the stable Vicuna seems to have the same issue -> https://huggingface.co/CarperAI/stable-vicuna-13b-delta/discussions/3
Yeah, that was the issue. They used Vicuna v0 which didn't train with a stopping token, and so it would only work with a specific prompt template. In my testing, changing special_tokens_map.json doesn't help, because it wasn't trained to use a stopping token in the first place.
So I'd prefer to keep my files matching the ones they released, just to avoid confusion.
Thanks anyway.
Just to say that CarperAI have just merged a change change similar to this, so you were right that it makes a difference to have this!
I will use the exact file they put into their repo, which has a couple of other changes as always. But just wanted to say thanks for PRing this.