Can this be used on VLLM?
#1
by
xxxant
- opened
I am working hard to find a quantitative model that can run on VLLM. Thanks!
Try it, it might work. If you have internet issues, try with small model first, if it works, try this one