Create README.md

README.md

@@ -0,0 +1,94 @@
+---
+license: llama3
+library_name: transformers
+tags:
+- nsfw
+- not-for-all-audiences
+- llama-3
+- text-generation-inference
+- moe
+- mergekit
+- merge
+---
+
+# Llama-Salad-4x8B-V3
+Changes in V3:
+- Uses `L3-8B-Stheno-v3.2` as the base model instead of `Meta-Llama-3-8B-Instruct`
+- Removed `opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5` and added `Einstein-v6.1-Llama3-8B`
+- Swapped `Llama-3-Soliloquy-8B-v2` for `L3-8B-Stheno-v3.2`
+
+# Details
+- **License**: [llama3](https://llama.meta.com/llama3/license/)
+- **Instruct Format**: [llama-3](https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/)
+- **Context Size**: 8K
+
+## Models Used
+- [Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct)
+- [llama-3-cat-8b-instruct-v1](https://huggingface.co/TheSkullery/llama-3-cat-8b-instruct-v1)
+- [Llama-3-8B-Synthia-v3.5](https://huggingface.co/migtissera/Llama-3-8B-Synthia-v3.5)
+- [Einstein-v6.1-Llama3-8B](https://huggingface.co/Weyaxi/Einstein-v6.1-Llama3-8B)
+
+## Merge Config
+```yaml
+base_model: Sao10K/L3-8B-Stheno-v3.2
+gate_mode: hidden
+dtype: bfloat16
+experts_per_token: 2
+experts:
+  - source_model: NousResearch/Meta-Llama-3-8B-Instruct
+    positive_prompts:
+    - "chat"
+    - "conversation"
+  - source_model: Weyaxi/Einstein-v6.1-Llama3-8B
+    positive_prompts:
+    - "science"
+    - "physics"
+    - "chemistry"
+    - "biology"
+    - "math"
+    - "step-by-step"
+    - "logical reasoning"
+    - "multilingual"
+    - "translation"
+    - "language translation"
+    - "foreign language"
+    negative_prompts:
+    - "programming language"
+  - source_model: migtissera/Llama-3-8B-Synthia-v3.5
+    positive_prompts:
+    - "summarize"
+    - "paraphrase"
+    - "list"
+    - "explain"
+    - "define"
+    - "analyze"
+    - "rephrase"
+    - "elaborate"
+    - "programming language"
+    - "JavaScript"
+    - "Python programming language"
+    - "Rust programming language"
+    - "C++ programming language"
+    - "GO programming language"
+    - "Ruby programming language"
+    - "Haskell programming language"
+    - "SQL query language"
+    - "CSS markup styling language"
+    - "code"
+  - source_model: Sao10K/L3-8B-Stheno-v3.2
+    positive_prompts:
+    - "characters"
+    - "scene"
+    - "roleplay"
+    - "erotic roleplay"
+    - "sexual fetish"
+    - "NSFW"
+    - "creative writing"
+    - "storytelling"
+    - "narration"
+    - "narrative setting"
+    - "narrative plot"
+    - "narrative exposition"
+    - "narrative theme"
+    - "narrative climax"
+```