Update README.md
Browse files
README.md
CHANGED
|
@@ -5,7 +5,11 @@ license: apache-2.0
|
|
| 5 |
|
| 6 |
# Model Card for Model ID
|
| 7 |
|
| 8 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
|
| 10 |
|
| 11 |
|
|
|
|
| 5 |
|
| 6 |
# Model Card for Model ID
|
| 7 |
|
| 8 |
+
This tiny model is an 81 million perimeter GPT2 based model it was trained from scratch on the 3060 TI. it uses the GPT2 tokenizer from the GPT2 repo here on hugging face.
|
| 9 |
+
|
| 10 |
+
We are training our own tokenizer from scratch and will release a version 2 of this trained on even more data sets once that is complete.
|
| 11 |
+
|
| 12 |
+
This model is in float 32 but will be converted shortly two float16 in bfloat16.
|
| 13 |
|
| 14 |
|
| 15 |
|