File size: 4,701 Bytes
541f773 f94c633 541f773 f94c633 f1e94b5 f94c633 1db424f 25ef053 f94c633 125b5df ed441fb 7c0ea99 b855bba b87205e f94c633 b87205e f94c633 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 |
---
library_name: transformers
tags:
- llama-factory
license: llama3
datasets:
- TheSkullery/Aether-Lite-V1.2
---
<!DOCTYPE html>
<style>
body {
font-family: 'Quicksand', sans-serif;
background: linear-gradient(135deg, #2E3440 0%, #1A202C 100%);
color: #D8DEE9;
margin: 0;
padding: 0;
font-size: 16px;
}
.container {
width: 80% auto;
max-width: 1080px auto;
margin: 20px auto;
background-color: rgba(255, 255, 255, 0.02);
padding: 20px;
border-radius: 12px;
box-shadow: 0 4px 10px rgba(0, 0, 0, 0.2);
backdrop-filter: blur(10px);
border: 1px solid rgba(255, 255, 255, 0.1);
}
.header h1 {
font-size: 28px;
color: #ECEFF4;
margin: 0 0 20px 0;
text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.3);
}
.update-section {
margin-top: 30px;
}
.update-section h2 {
font-size: 24px;
color: #88C0D0;
}
.update-section p {
font-size: 16px;
line-height: 1.6;
color: #ECEFF4;
}
.info img {
width: 100%;
border-radius: 10px;
margin-bottom: 15px;
}
a {
color: #88C0D0;
text-decoration: none;
}
a:hover {
color: #A3BE8C;
}
.button {
display: inline-block;
background-color: #5E81AC;
color: #E5E9F0;
padding: 10px 20px;
border-radius: 5px;
cursor: pointer;
text-decoration: none;
}
.button:hover {
background-color: #81A1C1;
}
pre {
background-color: #2E3440;
padding: 10px;
border-radius: 5px;
overflow-x: auto;
}
code {
font-family: 'Courier New', monospace;
color: #D8DEE9;
}
</style>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>L3-Aethora-15B Data Card</title>
<link href="https://fonts.googleapis.com/css2?family=Quicksand:wght@400;500;600&display=swap" rel="stylesheet">
</head>
<body>
<div class="container">
<div class="header">
<h1>L3-Aethora-15B</h1>
</div>
<div class="info">
<img src="https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/W0qzZK_V1Zt1GdgCIsnrP.png">
<p>The Skullery Presents L3-Aethora-15B.</p>
<p><strong>Creator:</strong> <a href="https://huggingface.co/steelskull" target="_blank">Steelskull</a></p>
<p><strong>Dataset:</strong> <a href="https://huggingface.co/datasets/TheSkullery/Aether-Lite-V1.2" target="_blank">Aether-Lite-V1.2</a></p>
<p><strong>Trained:</strong> 4 x A100 for 15 hours Using RsLora and DORA</p>
<h1>About L3-Aethora-15B:</h1>
<pre><code> L3 = Llama3 </code></pre>
<p>L3-Aethora-15B was crafted through using the abilteration method to adjust model responses. The model's refusal is inhibited, focusing on yielding more compliant and facilitative dialogue interactions. It then underwent a modified DUS (Depth Up Scale) merge (originally used by @Elinas) by using passthrough merge to create a 15b model, with specific adjustments (zeroing) to 'o_proj' and 'down_proj', enhancing its efficiency and reducing perplexity. This created AbL3In-15b.<br>
<p>AbL3In-15b was then trained for 4 epochs using Rslora & DORA training methods on the Aether-Lite-V1.2 dataset, containing ~82000 high quality samples, designed to strike a fine balance between creativity, slop, and intelligence at about a 60/40 split</p>
<p>This model is trained on the L3 prompt format.</p>
<h2>Quants:</h2>
<li><a href="https://huggingface.co/mradermacher/L3-Aethora-15B-GGUF" target="_blank">Mradermacher/L3-Aethora-15B-GGUF</a></li>
<li><a href="https://huggingface.co/NikolayKozloff" target="_blank">NikolayKozloff/L3-Aethora-15B-GGUF</a></li>
<p></p>
<h2>Dataset Summary: (Filtered)</h2>
<p>Filtered Phrases: GPTslop, Claudism's</p>
<ul>
<li><strong>mrfakename/Pure-Dove-ShareGPT:</strong> Processed 3707, Removed 150</li>
<li><strong>mrfakename/Capybara-ShareGPT:</strong> Processed 13412, Removed 2594</li>
<li><strong>jondurbin/airoboros-3.2:</strong> Processed 54517, Removed 4192</li>
<li><strong>PJMixers/grimulkan_theory-of-mind-ShareGPT:</strong> Processed 533, Removed 6</li>
<li><strong>grimulkan/PIPPA-augmented-dedup:</strong> Processed 869, Removed 46</li>
<li><strong>grimulkan/LimaRP-augmented:</strong> Processed 790, Removed 14</li>
<li><strong>PJMixers/grimulkan_physical-reasoning-ShareGPT:</strong> Processed 895, Removed 4</li>
<li><strong>MinervaAI/Aesir-Preview:</strong> Processed 994, Removed 6</li>
<li><strong>Doctor-Shotgun/no-robots-sharegpt:</strong> Processed 9911, Removed 89</li>
</ul>
<h2>Deduplication Stats:</h2>
<p>Starting row count: 85628, Final row count: 81960, Rows removed: 3668</p>
</div>
</div>
</body>
</html> |