Steelskull
commited on
Commit
•
25ef053
1
Parent(s):
125b5df
Update README.md
Browse files
README.md
CHANGED
@@ -110,11 +110,11 @@ code {
|
|
110 |
<p>The Skullery Presents L3-Aethora-15B.</p>
|
111 |
<p><strong>Creator:</strong> <a href="https://huggingface.co/steelskull" target="_blank">Steelskull</a></p>
|
112 |
<p><strong>Dataset:</strong> <a href="https://huggingface.co/datasets/TheSkullery/Aether-Lite-V1.2" target="_blank">Aether-Lite-V1.2</a></p>
|
113 |
-
<p><strong>Trained:</strong> 4 x A100 for 15 hours</p>
|
114 |
<h1>About L3-Aethora-15B:</h1>
|
115 |
<pre><code> L3 = Llama3 </code></pre>
|
116 |
-
<p>L3-Aethora-15B was crafted through using the abilteration method to adjust model responses. The model's refusal is inhibited, focusing on yielding more compliant and facilitative dialogue interactions. It underwent a
|
117 |
-
<p>AbL3In-15b was then
|
118 |
<p>Filtered Phrases: GPTslop, Claudism's</p>
|
119 |
<h2>Dataset Summary: (Filtered)</h2>
|
120 |
<ul>
|
|
|
110 |
<p>The Skullery Presents L3-Aethora-15B.</p>
|
111 |
<p><strong>Creator:</strong> <a href="https://huggingface.co/steelskull" target="_blank">Steelskull</a></p>
|
112 |
<p><strong>Dataset:</strong> <a href="https://huggingface.co/datasets/TheSkullery/Aether-Lite-V1.2" target="_blank">Aether-Lite-V1.2</a></p>
|
113 |
+
<p><strong>Trained:</strong> 4 x A100 for 15 hours Using RsLora and DORA</p>
|
114 |
<h1>About L3-Aethora-15B:</h1>
|
115 |
<pre><code> L3 = Llama3 </code></pre>
|
116 |
+
<p>L3-Aethora-15B was crafted through using the abilteration method to adjust model responses. The model's refusal is inhibited, focusing on yielding more compliant and facilitative dialogue interactions. It then underwent a modified DUS (Depth Up Scale) merge (originally used by @Elias) by using passthrough merge to create a 15b model, with specific adjustments (zeroing) to 'o_proj' and 'down_proj', enhancing its efficiency and reducing perplexity. This created AbL3In-15b.<br>
|
117 |
+
<p>AbL3In-15b was then trained for 4 epochs using Rslora & DORA training methods on the Aether-Lite-V1 dataset, containing ~82000 high quality samples, designed to strike a fine balance between creativity, slop, and intelligence at about a 60/40 split</p>
|
118 |
<p>Filtered Phrases: GPTslop, Claudism's</p>
|
119 |
<h2>Dataset Summary: (Filtered)</h2>
|
120 |
<ul>
|