Chat template
#1
by
Handgun1773
- opened
As in the original repo, the chat template in your GGUF doesn't match the one described in the original model card.
I made a PR in the original repo.
Thanks for the heads up! Let me know when it gets merged and I'll remake this with the proper template
@Handgun1773 am I understanding this correctly, the line you changed in the original repo would be applied when running the model with llama.cpp?
Is the benefit of this that you can just simply use:
-p "write me a story"
As opposed to needing to add prefixes and all those flags?
I don't know if llama.cpp take into account prompt templates from the GGUF files. I know ooba does when loading a GGUF with llama.cpp backend.