sophosympatheia
commited on
Commit
•
c0b298d
1
Parent(s):
d862b4f
Update README.md
Browse filesAdded links to more quants
README.md
CHANGED
@@ -121,9 +121,15 @@ If you save this as a .json file, you can import it directly into Silly Tavern.
|
|
121 |
This repo contains branches for various exllama2 quanizations of the model calibratend on a version of the PIPPA dataset.
|
122 |
|
123 |
* Main Branch, Full weights
|
124 |
-
* 3.2 bpw -- This will fit comfortably within 48 GB of VRAM at 8192 context.
|
125 |
-
* 3.35 bpw
|
126 |
-
* 3.
|
|
|
|
|
|
|
|
|
|
|
|
|
127 |
|
128 |
### Licence and usage restrictions
|
129 |
|
|
|
121 |
This repo contains branches for various exllama2 quanizations of the model calibratend on a version of the PIPPA dataset.
|
122 |
|
123 |
* Main Branch, Full weights
|
124 |
+
* 3.2 bpw branch -- This will fit comfortably within 48 GB of VRAM at 8192 context.
|
125 |
+
* 3.35 bpw branch -- This will fit within 48 GB of VRAM at 4096 context without using the 8-bit cache setting.
|
126 |
+
* 3.0 bpw -- [LoneStriker/Rogue-Rose-103b-v0.2-3.0bpw-h6-exl2-2](https://huggingface.co/LoneStriker/Rogue-Rose-103b-v0.2-3.0bpw-h6-exl2-2)
|
127 |
+
* 4.0 bpw -- [LoneStriker/Rogue-Rose-103b-v0.2-4.0bpw-h6-exl2-2](https://huggingface.co/LoneStriker/Rogue-Rose-103b-v0.2-4.0bpw-h6-exl2-2)
|
128 |
+
* 5.0 bpw -- [LoneStriker/Rogue-Rose-103b-v0.2-5.0bpw-h6-exl2-2](https://huggingface.co/LoneStriker/Rogue-Rose-103b-v0.2-5.0bpw-h6-exl2-2)
|
129 |
+
* GGUF -- [TheBloke/Rogue-Rose-103b-v0.2-GGUF](https://huggingface.co/TheBloke/Rogue-Rose-103b-v0.2-GGUF)
|
130 |
+
* AWQ -- [TheBloke/Rogue-Rose-103b-v0.2-AWQ](https://huggingface.co/TheBloke/Rogue-Rose-103b-v0.2-AWQ)
|
131 |
+
* GPTQ -- [TheBloke/Rogue-Rose-103b-v0.2-GPTQ](https://huggingface.co/TheBloke/Rogue-Rose-103b-v0.2-GPTQ)
|
132 |
+
|
133 |
|
134 |
### Licence and usage restrictions
|
135 |
|