Update README.md
Browse files
README.md
CHANGED
|
@@ -92,10 +92,18 @@ repetition_penalty: 1.17
|
|
| 92 |
top_k: 49
|
| 93 |
```
|
| 94 |
|
| 95 |
-
## Quantized
|
|
|
|
|
|
|
| 96 |
|
| 97 |
A 4.250b EXL2 version of the model can be found here:
|
| 98 |
|
| 99 |
https://huggingface.co/oobabooga/CodeBooga-34B-v0.1-EXL2-4.250b
|
| 100 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
<a href="https://ko-fi.com/oobabooga"><img src="https://i.imgur.com/UJlEAYw.png"></a>
|
|
|
|
| 92 |
top_k: 49
|
| 93 |
```
|
| 94 |
|
| 95 |
+
## Quantized versions
|
| 96 |
+
|
| 97 |
+
### EXL2
|
| 98 |
|
| 99 |
A 4.250b EXL2 version of the model can be found here:
|
| 100 |
|
| 101 |
https://huggingface.co/oobabooga/CodeBooga-34B-v0.1-EXL2-4.250b
|
| 102 |
|
| 103 |
+
### GGUF
|
| 104 |
+
|
| 105 |
+
TheBloke has kindly provided GGUF quantizations for llama.cpp:
|
| 106 |
+
|
| 107 |
+
https://huggingface.co/TheBloke/CodeBooga-34B-v0.1-GGUF
|
| 108 |
+
|
| 109 |
<a href="https://ko-fi.com/oobabooga"><img src="https://i.imgur.com/UJlEAYw.png"></a>
|