Configuring command-r-gptq

#33
by Cyleux - opened

I made a GPT-Q version of command-r but am having trouble running it. Is there something special to do for quantization?

https://huggingface.co/Cyleux/command-r-gptq

Would anyone be willing to help fix it or release one?

Sign up or log in to comment