metadata
license: llama3.1
EXL2 quants of Llama-3.1 70B Instruct
This model requires the dev branch of ExLlamaV2 for now. New release coming soon with the necessary changes.
2.50 bits per weight
3.00 bits per weight
3.50 bits per weight
4.00 bits per weight
4.50 bits per weight
5.00 bits per weight
6.00 bits per weight