Commit
•
29b2c22
1
Parent(s):
71195ef
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,16 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
I cut my TinyLlama 1.1B cinder v 2 down from 22 layers to 14. At 14 there was no coherent text but there were emerging ideas of a response. 1000 steps on step-by-step dataset.
|
| 5 |
6000 on Reason-with-cinder. The loss was still over 1 and the learning rate was still over 4. This model needs significat training. I am putting it up as a base model that
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
widget:
|
| 4 |
+
- text: >
|
| 5 |
+
<|system|>
|
| 6 |
+
|
| 7 |
+
You are a helpful assistant</s>
|
| 8 |
+
|
| 9 |
+
<|user|>
|
| 10 |
+
|
| 11 |
+
Tell me about yourself, what is your name?</s>
|
| 12 |
+
|
| 13 |
+
<|assistant|>
|
| 14 |
---
|
| 15 |
I cut my TinyLlama 1.1B cinder v 2 down from 22 layers to 14. At 14 there was no coherent text but there were emerging ideas of a response. 1000 steps on step-by-step dataset.
|
| 16 |
6000 on Reason-with-cinder. The loss was still over 1 and the learning rate was still over 4. This model needs significat training. I am putting it up as a base model that
|