Update README.md
#1
by
Dampfinchen
- opened
README.md
CHANGED
@@ -11,7 +11,7 @@ base_model:
|
|
11 |
tags:
|
12 |
- mergekit
|
13 |
- merge
|
14 |
-
|
15 |
---
|
16 |
# mergeout
|
17 |
|
@@ -66,3 +66,16 @@ dtype: bfloat16
|
|
66 |
name: Mega-Destroyer-8x7B
|
67 |
|
68 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
11 |
tags:
|
12 |
- mergekit
|
13 |
- merge
|
14 |
+
license: cc-by-4.0
|
15 |
---
|
16 |
# mergeout
|
17 |
|
|
|
66 |
name: Mega-Destroyer-8x7B
|
67 |
|
68 |
```
|
69 |
+
|
70 |
+
Hello everyone, this is Dampf. You might know me as the creator of Mythical-Destroyer-13B.
|
71 |
+
|
72 |
+
This time, I collaborated with Mr.DragonFox aka FoxEngineAi, harnessing his powerful rig to deliver a Merge of multiple high quality Mixtral 8x7B models. My goal was to beat Bagel-Mistery-Tour V2 by Ycros and create the best Mixtral model to date. Did I succeed? Please try it out and decide for yourself!
|
73 |
+
|
74 |
+
Aside from the obvious Mixtral Instruct, to keep its intelligence, I've merged Rombo's excellent Open_Gpt4_v0.2 model that consists of Jon Durbin's Bagel-DPO-8x7B and another highly regarded model, namely smelborp/MixtralOrochi8x7B. This model also combines different datasets together, meaning it should be agood fit for every task you throw at it. This model acts like the reasoning part in the merge.
|
75 |
+
In contrast, we have Air-Striker and LimaRP at the creative side which will allow for great roleplays in different styles, they are also a good fit to enhance the model's writing capabilities greatly.
|
76 |
+
|
77 |
+
And finally, I've merged Sao10K/Typhon-Mixtral-v1 to boost the story writing capabilities even further. It includes KoboldAI's latest Holodeck model, as well as a couple of his latest models and combines it into one package. My hope is that this will capture the magic Sao10K/Fimbulvetr-11B-v2 emits, just at the intelligence level of a Mixtral model. This one also includes Nous Hermes 2 DPO, a high quality instruct model that will boost its intelligence and sorta act like a balancer to all the creative stuff in the merge.
|
78 |
+
|
79 |
+
What we have here is a model that should be fantastic at instruct and roleplay/creative tasks a like. So basically a general purpose model. Perhaps the pinnacle of Rocksmashing? Idk xD I just know it includes nearly all datasets on the sun. As a reason, it will likely work with every prompt format as well. So feel free to use Alpaca, Vicuna, ChatML, Llama 2 Chat or whatever your heart desires.
|
80 |
+
|
81 |
+
A huge thank you to the creators of these fantastic datasets and fine tunes in the respective merges, namely Jon Durbin, Teknium, Sao10K, MistralAI, LoneStriker, NeverSleep, Suikamelon, Doctor-Shotgun, KoboldAI and more. All credit goes to them. A thank you to the creators of the different merges I've merged (Mergeception!) as well! And of course a thank you to MrDragonFox for lending his compute! Please enjoy :D
|