RichardErkhov commited on
Commit
8dbcc86
1 Parent(s): 09e9412

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +203 -0
README.md ADDED
@@ -0,0 +1,203 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ Prima-LelantaclesV5-7b - GGUF
11
+ - Model creator: https://huggingface.co/ChaoticNeutrals/
12
+ - Original model: https://huggingface.co/ChaoticNeutrals/Prima-LelantaclesV5-7b/
13
+
14
+
15
+ | Name | Quant method | Size |
16
+ | ---- | ---- | ---- |
17
+ | [Prima-LelantaclesV5-7b.Q2_K.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q2_K.gguf) | Q2_K | 2.53GB |
18
+ | [Prima-LelantaclesV5-7b.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.IQ3_XS.gguf) | IQ3_XS | 2.81GB |
19
+ | [Prima-LelantaclesV5-7b.IQ3_S.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.IQ3_S.gguf) | IQ3_S | 2.96GB |
20
+ | [Prima-LelantaclesV5-7b.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q3_K_S.gguf) | Q3_K_S | 2.95GB |
21
+ | [Prima-LelantaclesV5-7b.IQ3_M.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.IQ3_M.gguf) | IQ3_M | 3.06GB |
22
+ | [Prima-LelantaclesV5-7b.Q3_K.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q3_K.gguf) | Q3_K | 3.28GB |
23
+ | [Prima-LelantaclesV5-7b.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q3_K_M.gguf) | Q3_K_M | 3.28GB |
24
+ | [Prima-LelantaclesV5-7b.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q3_K_L.gguf) | Q3_K_L | 3.56GB |
25
+ | [Prima-LelantaclesV5-7b.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.IQ4_XS.gguf) | IQ4_XS | 3.67GB |
26
+ | [Prima-LelantaclesV5-7b.Q4_0.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q4_0.gguf) | Q4_0 | 3.83GB |
27
+ | [Prima-LelantaclesV5-7b.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.IQ4_NL.gguf) | IQ4_NL | 3.87GB |
28
+ | [Prima-LelantaclesV5-7b.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q4_K_S.gguf) | Q4_K_S | 3.86GB |
29
+ | [Prima-LelantaclesV5-7b.Q4_K.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q4_K.gguf) | Q4_K | 4.07GB |
30
+ | [Prima-LelantaclesV5-7b.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q4_K_M.gguf) | Q4_K_M | 4.07GB |
31
+ | [Prima-LelantaclesV5-7b.Q4_1.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q4_1.gguf) | Q4_1 | 4.24GB |
32
+ | [Prima-LelantaclesV5-7b.Q5_0.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q5_0.gguf) | Q5_0 | 4.65GB |
33
+ | [Prima-LelantaclesV5-7b.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q5_K_S.gguf) | Q5_K_S | 4.65GB |
34
+ | [Prima-LelantaclesV5-7b.Q5_K.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q5_K.gguf) | Q5_K | 4.78GB |
35
+ | [Prima-LelantaclesV5-7b.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q5_K_M.gguf) | Q5_K_M | 4.78GB |
36
+ | [Prima-LelantaclesV5-7b.Q5_1.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q5_1.gguf) | Q5_1 | 5.07GB |
37
+ | [Prima-LelantaclesV5-7b.Q6_K.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q6_K.gguf) | Q6_K | 5.53GB |
38
+ | [Prima-LelantaclesV5-7b.Q8_0.gguf](https://huggingface.co/RichardErkhov/ChaoticNeutrals_-_Prima-LelantaclesV5-7b-gguf/blob/main/Prima-LelantaclesV5-7b.Q8_0.gguf) | Q8_0 | 7.17GB |
39
+
40
+
41
+
42
+
43
+ Original model description:
44
+ ---
45
+ license: other
46
+ library_name: transformers
47
+ tags:
48
+ - mergekit
49
+ - merge
50
+ base_model:
51
+ - Test157t/Pasta-Lake-7b
52
+ - Test157t/Prima-LelantaclesV4-7b-16k
53
+ model-index:
54
+ - name: Prima-LelantaclesV5-7b
55
+ results:
56
+ - task:
57
+ type: text-generation
58
+ name: Text Generation
59
+ dataset:
60
+ name: AI2 Reasoning Challenge (25-Shot)
61
+ type: ai2_arc
62
+ config: ARC-Challenge
63
+ split: test
64
+ args:
65
+ num_few_shot: 25
66
+ metrics:
67
+ - type: acc_norm
68
+ value: 70.65
69
+ name: normalized accuracy
70
+ source:
71
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ChaoticNeutrals/Prima-LelantaclesV5-7b
72
+ name: Open LLM Leaderboard
73
+ - task:
74
+ type: text-generation
75
+ name: Text Generation
76
+ dataset:
77
+ name: HellaSwag (10-Shot)
78
+ type: hellaswag
79
+ split: validation
80
+ args:
81
+ num_few_shot: 10
82
+ metrics:
83
+ - type: acc_norm
84
+ value: 87.87
85
+ name: normalized accuracy
86
+ source:
87
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ChaoticNeutrals/Prima-LelantaclesV5-7b
88
+ name: Open LLM Leaderboard
89
+ - task:
90
+ type: text-generation
91
+ name: Text Generation
92
+ dataset:
93
+ name: MMLU (5-Shot)
94
+ type: cais/mmlu
95
+ config: all
96
+ split: test
97
+ args:
98
+ num_few_shot: 5
99
+ metrics:
100
+ - type: acc
101
+ value: 64.52
102
+ name: accuracy
103
+ source:
104
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ChaoticNeutrals/Prima-LelantaclesV5-7b
105
+ name: Open LLM Leaderboard
106
+ - task:
107
+ type: text-generation
108
+ name: Text Generation
109
+ dataset:
110
+ name: TruthfulQA (0-shot)
111
+ type: truthful_qa
112
+ config: multiple_choice
113
+ split: validation
114
+ args:
115
+ num_few_shot: 0
116
+ metrics:
117
+ - type: mc2
118
+ value: 68.26
119
+ source:
120
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ChaoticNeutrals/Prima-LelantaclesV5-7b
121
+ name: Open LLM Leaderboard
122
+ - task:
123
+ type: text-generation
124
+ name: Text Generation
125
+ dataset:
126
+ name: Winogrande (5-shot)
127
+ type: winogrande
128
+ config: winogrande_xl
129
+ split: validation
130
+ args:
131
+ num_few_shot: 5
132
+ metrics:
133
+ - type: acc
134
+ value: 82.4
135
+ name: accuracy
136
+ source:
137
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ChaoticNeutrals/Prima-LelantaclesV5-7b
138
+ name: Open LLM Leaderboard
139
+ - task:
140
+ type: text-generation
141
+ name: Text Generation
142
+ dataset:
143
+ name: GSM8k (5-shot)
144
+ type: gsm8k
145
+ config: main
146
+ split: test
147
+ args:
148
+ num_few_shot: 5
149
+ metrics:
150
+ - type: acc
151
+ value: 64.82
152
+ name: accuracy
153
+ source:
154
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ChaoticNeutrals/Prima-LelantaclesV5-7b
155
+ name: Open LLM Leaderboard
156
+ ---
157
+ Update: Getting suprisingly good results at 16384 context, which is unexpected given this context pool should remain untouched from other mistral models working around 8192.
158
+
159
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/642265bc01c62c1e4102dc36/iZWd2VINrrl-ToMoD9ZUp.png)
160
+
161
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/642265bc01c62c1e4102dc36/_AugGaelWylUuIIDmYOXG.jpeg)
162
+
163
+ Thanks to @Lewdiculus for the Quants: https://huggingface.co/Lewdiculous/Prima-LelantaclesV5-7b-GGUF
164
+
165
+ This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method.
166
+
167
+ The following models were included in the merge:
168
+ * [Test157t/Pasta-Lake-7b](https://huggingface.co/Test157t/Pasta-Lake-7b) + [Test157t/Prima-LelantaclesV4-7b-16k](https://huggingface.co/Test157t/Prima-LelantaclesV4-7b-16k)
169
+
170
+ ### Configuration
171
+
172
+ The following YAML configuration was used to produce this model:
173
+
174
+ ```yaml
175
+ merge_method: dare_ties
176
+ base_model: Test157t/Prima-LelantaclesV4-7b-16k
177
+ parameters:
178
+ normalize: true
179
+ models:
180
+ - model: Test157t/Pasta-Lake-7b
181
+ parameters:
182
+ weight: 1
183
+ - model: Test157t/Prima-LelantaclesV4-7b-16k
184
+ parameters:
185
+ weight: 1
186
+ dtype: float16
187
+
188
+ ```
189
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
190
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ChaoticNeutrals__Prima-LelantaclesV5-7b)
191
+
192
+ | Metric |Value|
193
+ |---------------------------------|----:|
194
+ |Avg. |73.09|
195
+ |AI2 Reasoning Challenge (25-Shot)|70.65|
196
+ |HellaSwag (10-Shot) |87.87|
197
+ |MMLU (5-Shot) |64.52|
198
+ |TruthfulQA (0-shot) |68.26|
199
+ |Winogrande (5-shot) |82.40|
200
+ |GSM8k (5-shot) |64.82|
201
+
202
+
203
+