missing config.json?
Trying to run it GGML version with oobaabooga, and fails missing config.json?
Does ooba even support GGML models? The only file llama.cpp needs to work is the .bin file itself.
yes ooba supports GGML models. I only have 8 GB VRAM so can only run 13b models when they are GGML.
The model works great on llama.cpp and koboldCpp. The version of llama.cpp included in ooba might be too outdated to use the newer 5.1 quantization method, or maybe you're on an older version? I imagine they'll update it soon-ish if that's the case.
Found a fix to get the model wowroking with oobaabooga.
The model has to have "ggml" in the filename. Simply rename the model with those letters in there and it works.
found this in the oobaabooga llama.cpp instructions.
I still have the same problem after renaming the file to "ggml". But I did manage to get Koboldcpp to work. Still would prefer ooba though.
You can use KoboldCpp as a backend, then use a custom front-end such as TavernAI or whatever. The only difference is that the default KoboldCpp port is :5001 instead of :5000, but you can change it to whatever you like with a launch parameter.
mithrid was right. I made a mistake and changed the "bin" part of the file name to "ggml". I should have just left it as a bin file and just added "ggml" somewhere like "WizardML-Unc-13b-Q5_1.ggml.bin". It's working now.