neph1

/

bellman-7b-mistral-instruct-v0.2

like 11

Thanks!
Would a Q3 quant do( 3.5GB)? I tested a newer version as Q3 and hardly noticed any difference between it and the Q8(!). I don't mind finetuning a 3b, but the tricky part is finding one that has somewhat decent Swedish.

That would be cool to try out! :)

I hope to have a new version up in a couple of days. I'll make a Q3 quant for it.

There's a now a q3_k_m with the latest quantization stuff from llama.cpp. Let me know how it performs.