Commit
•
2a0a31c
1
Parent(s):
fb43674
Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
datasets:
|
| 4 |
+
- togethercomputer/RedPajama-Data-1T-Sample
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
+
library_name: transformers
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
This is a set of sparse autoencoders (SAEs) trained on the residual stream of [Llama 3 8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) using the [RedPajama corpus](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T-Sample). The SAEs are organized by layer, and can be loaded using the EleutherAI [`sae` library](https://github.com/EleutherAI/sae).
|