Upload README.md
Browse files
README.md
CHANGED
|
@@ -383,12 +383,20 @@ repetition_penalty: 1.17
|
|
| 383 |
top_k: 49
|
| 384 |
```
|
| 385 |
|
| 386 |
-
## Quantized
|
|
|
|
|
|
|
| 387 |
|
| 388 |
A 4.250b EXL2 version of the model can be found here:
|
| 389 |
|
| 390 |
https://huggingface.co/oobabooga/CodeBooga-34B-v0.1-EXL2-4.250b
|
| 391 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 392 |
<a href="https://ko-fi.com/oobabooga"><img src="https://i.imgur.com/UJlEAYw.png"></a>
|
| 393 |
|
| 394 |
<!-- original-model-card end -->
|
|
|
|
| 383 |
top_k: 49
|
| 384 |
```
|
| 385 |
|
| 386 |
+
## Quantized versions
|
| 387 |
+
|
| 388 |
+
### EXL2
|
| 389 |
|
| 390 |
A 4.250b EXL2 version of the model can be found here:
|
| 391 |
|
| 392 |
https://huggingface.co/oobabooga/CodeBooga-34B-v0.1-EXL2-4.250b
|
| 393 |
|
| 394 |
+
### GGUF
|
| 395 |
+
|
| 396 |
+
TheBloke has kindly provided GGUF quantizations for llama.cpp:
|
| 397 |
+
|
| 398 |
+
https://huggingface.co/TheBloke/CodeBooga-34B-v0.1-GGUF
|
| 399 |
+
|
| 400 |
<a href="https://ko-fi.com/oobabooga"><img src="https://i.imgur.com/UJlEAYw.png"></a>
|
| 401 |
|
| 402 |
<!-- original-model-card end -->
|