RichardErkhov commited on
Commit
3a2a4d7
1 Parent(s): bb08bb9

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +166 -0
README.md ADDED
@@ -0,0 +1,166 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ Buttocks-7B-v1.1 - bnb 8bits
11
+ - Model creator: https://huggingface.co/TeeZee/
12
+ - Original model: https://huggingface.co/TeeZee/Buttocks-7B-v1.1/
13
+
14
+
15
+
16
+
17
+ Original model description:
18
+ ---
19
+ license: cc-by-nc-4.0
20
+ tags:
21
+ - not-for-all-audiences
22
+ - merge
23
+ model-index:
24
+ - name: Buttocks-7B-v1.1
25
+ results:
26
+ - task:
27
+ type: text-generation
28
+ name: Text Generation
29
+ dataset:
30
+ name: AI2 Reasoning Challenge (25-Shot)
31
+ type: ai2_arc
32
+ config: ARC-Challenge
33
+ split: test
34
+ args:
35
+ num_few_shot: 25
36
+ metrics:
37
+ - type: acc_norm
38
+ value: 54.61
39
+ name: normalized accuracy
40
+ source:
41
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/Buttocks-7B-v1.1
42
+ name: Open LLM Leaderboard
43
+ - task:
44
+ type: text-generation
45
+ name: Text Generation
46
+ dataset:
47
+ name: HellaSwag (10-Shot)
48
+ type: hellaswag
49
+ split: validation
50
+ args:
51
+ num_few_shot: 10
52
+ metrics:
53
+ - type: acc_norm
54
+ value: 75.61
55
+ name: normalized accuracy
56
+ source:
57
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/Buttocks-7B-v1.1
58
+ name: Open LLM Leaderboard
59
+ - task:
60
+ type: text-generation
61
+ name: Text Generation
62
+ dataset:
63
+ name: MMLU (5-Shot)
64
+ type: cais/mmlu
65
+ config: all
66
+ split: test
67
+ args:
68
+ num_few_shot: 5
69
+ metrics:
70
+ - type: acc
71
+ value: 50.22
72
+ name: accuracy
73
+ source:
74
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/Buttocks-7B-v1.1
75
+ name: Open LLM Leaderboard
76
+ - task:
77
+ type: text-generation
78
+ name: Text Generation
79
+ dataset:
80
+ name: TruthfulQA (0-shot)
81
+ type: truthful_qa
82
+ config: multiple_choice
83
+ split: validation
84
+ args:
85
+ num_few_shot: 0
86
+ metrics:
87
+ - type: mc2
88
+ value: 44.72
89
+ source:
90
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/Buttocks-7B-v1.1
91
+ name: Open LLM Leaderboard
92
+ - task:
93
+ type: text-generation
94
+ name: Text Generation
95
+ dataset:
96
+ name: Winogrande (5-shot)
97
+ type: winogrande
98
+ config: winogrande_xl
99
+ split: validation
100
+ args:
101
+ num_few_shot: 5
102
+ metrics:
103
+ - type: acc
104
+ value: 68.9
105
+ name: accuracy
106
+ source:
107
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/Buttocks-7B-v1.1
108
+ name: Open LLM Leaderboard
109
+ - task:
110
+ type: text-generation
111
+ name: Text Generation
112
+ dataset:
113
+ name: GSM8k (5-shot)
114
+ type: gsm8k
115
+ config: main
116
+ split: test
117
+ args:
118
+ num_few_shot: 5
119
+ metrics:
120
+ - type: acc
121
+ value: 5.76
122
+ name: accuracy
123
+ source:
124
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/Buttocks-7B-v1.1
125
+ name: Open LLM Leaderboard
126
+ ---
127
+
128
+ ### Buttocks 7B v1.1 ###
129
+
130
+ An experiment that has gone very, very wrong.
131
+
132
+ ### Model details ###
133
+
134
+ - Recreation of the original recipe for [Undi95/Toppy-M-7B](https://huggingface.co/Undi95/Toppy-M-7B), but instead of final merge done by mergekit, [MergeMoster](https://github.com/Gryphe/MergeMonster/) was used with extended RPG preset.
135
+ - recipe in [mergekit-config](https://huggingface.co/TeeZee/Toppy-7B-remake-mergemonster-SLERP-v1.0/resolve/main/toppy-slerp-merge-config.yml), stepsAA, BB, CC are the original models with LORAS as per Toppy M 7B sauce.
136
+ - LERP merge method was used
137
+
138
+ ### Results ###
139
+
140
+ - in simple terms this model is totally unhinged
141
+ - it always produces sequences similar to fever dreams or drug trips
142
+ - on a good day it can produce scenarios similar to old Monty Python sketches
143
+ - models shows incredible affinity to words like 'ass', 'buttocks', 'farts', prompting with those single words will probably
144
+ produce a whole story revolving around those topics.
145
+
146
+ ### Possible uses ###
147
+
148
+ - to generate dream sequence in a story
149
+ - to make the boring model more unpredictable by merging at low weights with this monster
150
+ - to take a break, connect Silly Tavern to this model and get a few ROTFLs observing how every story deteriorates into pure craziness
151
+ - research on LLM hallucinations
152
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
153
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_TeeZee__Buttocks-7B-v1.1)
154
+
155
+ | Metric |Value|
156
+ |---------------------------------|----:|
157
+ |Avg. |49.97|
158
+ |AI2 Reasoning Challenge (25-Shot)|54.61|
159
+ |HellaSwag (10-Shot) |75.61|
160
+ |MMLU (5-Shot) |50.22|
161
+ |TruthfulQA (0-shot) |44.72|
162
+ |Winogrande (5-shot) |68.90|
163
+ |GSM8k (5-shot) | 5.76|
164
+
165
+
166
+