Hugging Face's logo

saishf
/
SOVLish-Devil-8B-L3

Text Generation
Transformers
Safetensors
llama
mergekit
Merge
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
2
saishf leaderboard-pr-bot commited on
Commit
78d5349
•
1 Parent(s): 39d7a97

Adding Evaluation Results (#2)

Browse files

- Adding Evaluation Results (3371f293ab1fa2d765c956c5c6b873c61d93879c)


Co-authored-by: Open LLM Leaderboard PR Bot <[email protected]>

Files changed (1) hide show
  1. README.md +122 -6
README.md CHANGED
@@ -1,4 +1,9 @@
1
  ---
 
 
 
 
 
2
  base_model:
3
  - mlabonne/Daredevil-8B-abliterated
4
  - ResplendentAI/RP_Format_QuoteAsterisk_Llama3
@@ -11,11 +16,109 @@ base_model:
11
  - ResplendentAI/Aura_Llama3
12
  - mlabonne/Daredevil-8B-abliterated
13
  - ResplendentAI/Smarts_Llama3
14
- library_name: transformers
15
- tags:
16
- - mergekit
17
- - merge
18
- license: cc-by-nc-4.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  ---
20
  # merge
21
 
@@ -59,4 +162,17 @@ models:
59
  merge_method: model_stock
60
  base_model: mlabonne/Daredevil-8B-abliterated
61
  dtype: bfloat16
62
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: cc-by-nc-4.0
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
  base_model:
8
  - mlabonne/Daredevil-8B-abliterated
9
  - ResplendentAI/RP_Format_QuoteAsterisk_Llama3
 
16
  - ResplendentAI/Aura_Llama3
17
  - mlabonne/Daredevil-8B-abliterated
18
  - ResplendentAI/Smarts_Llama3
19
+ model-index:
20
+ - name: SOVLish-Devil-8B-L3
21
+ results:
22
+ - task:
23
+ type: text-generation
24
+ name: Text Generation
25
+ dataset:
26
+ name: AI2 Reasoning Challenge (25-Shot)
27
+ type: ai2_arc
28
+ config: ARC-Challenge
29
+ split: test
30
+ args:
31
+ num_few_shot: 25
32
+ metrics:
33
+ - type: acc_norm
34
+ value: 69.2
35
+ name: normalized accuracy
36
+ source:
37
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/SOVLish-Devil-8B-L3
38
+ name: Open LLM Leaderboard
39
+ - task:
40
+ type: text-generation
41
+ name: Text Generation
42
+ dataset:
43
+ name: HellaSwag (10-Shot)
44
+ type: hellaswag
45
+ split: validation
46
+ args:
47
+ num_few_shot: 10
48
+ metrics:
49
+ - type: acc_norm
50
+ value: 84.44
51
+ name: normalized accuracy
52
+ source:
53
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/SOVLish-Devil-8B-L3
54
+ name: Open LLM Leaderboard
55
+ - task:
56
+ type: text-generation
57
+ name: Text Generation
58
+ dataset:
59
+ name: MMLU (5-Shot)
60
+ type: cais/mmlu
61
+ config: all
62
+ split: test
63
+ args:
64
+ num_few_shot: 5
65
+ metrics:
66
+ - type: acc
67
+ value: 68.97
68
+ name: accuracy
69
+ source:
70
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/SOVLish-Devil-8B-L3
71
+ name: Open LLM Leaderboard
72
+ - task:
73
+ type: text-generation
74
+ name: Text Generation
75
+ dataset:
76
+ name: TruthfulQA (0-shot)
77
+ type: truthful_qa
78
+ config: multiple_choice
79
+ split: validation
80
+ args:
81
+ num_few_shot: 0
82
+ metrics:
83
+ - type: mc2
84
+ value: 57.95
85
+ source:
86
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/SOVLish-Devil-8B-L3
87
+ name: Open LLM Leaderboard
88
+ - task:
89
+ type: text-generation
90
+ name: Text Generation
91
+ dataset:
92
+ name: Winogrande (5-shot)
93
+ type: winogrande
94
+ config: winogrande_xl
95
+ split: validation
96
+ args:
97
+ num_few_shot: 5
98
+ metrics:
99
+ - type: acc
100
+ value: 78.14
101
+ name: accuracy
102
+ source:
103
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/SOVLish-Devil-8B-L3
104
+ name: Open LLM Leaderboard
105
+ - task:
106
+ type: text-generation
107
+ name: Text Generation
108
+ dataset:
109
+ name: GSM8k (5-shot)
110
+ type: gsm8k
111
+ config: main
112
+ split: test
113
+ args:
114
+ num_few_shot: 5
115
+ metrics:
116
+ - type: acc
117
+ value: 72.48
118
+ name: accuracy
119
+ source:
120
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/SOVLish-Devil-8B-L3
121
+ name: Open LLM Leaderboard
122
  ---
123
  # merge
124
 
 
162
  merge_method: model_stock
163
  base_model: mlabonne/Daredevil-8B-abliterated
164
  dtype: bfloat16
165
+ ```
166
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
167
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_saishf__SOVLish-Devil-8B-L3)
168
+
169
+ | Metric |Value|
170
+ |---------------------------------|----:|
171
+ |Avg. |71.86|
172
+ |AI2 Reasoning Challenge (25-Shot)|69.20|
173
+ |HellaSwag (10-Shot) |84.44|
174
+ |MMLU (5-Shot) |68.97|
175
+ |TruthfulQA (0-shot) |57.95|
176
+ |Winogrande (5-shot) |78.14|
177
+ |GSM8k (5-shot) |72.48|
178
+