chore: add info on dtypes
Browse files
README.md
CHANGED
|
@@ -31,3 +31,5 @@ StripedHyena is a hybrid architecture composed of multi-head, grouped-query atte
|
|
| 31 |
### Disclaimer
|
| 32 |
|
| 33 |
To use StripedHyena outside of the playground, you will need to install custom kernels. Please follow the instructions from the [standalone repository](https://github.com/togethercomputer/stripedhyena).
|
|
|
|
|
|
|
|
|
| 31 |
### Disclaimer
|
| 32 |
|
| 33 |
To use StripedHyena outside of the playground, you will need to install custom kernels. Please follow the instructions from the [standalone repository](https://github.com/togethercomputer/stripedhyena).
|
| 34 |
+
|
| 35 |
+
StripedHyena is a mixed precision model. Make sure to keep your `poles` and `residues` in `float32` precision, especially for longer prompts or training.
|