Mamba distilled from Zephyr. The Mamba in the Llama: Distilling and Accelerating Hybrid Models (https://arxiv.org/abs/2408.15237).
Junxiong Wang
JunxiongWang
AI & ML interests
Attention Free Model / Subquadratic Language Models
Organizations
Collections
4
models
28
JunxiongWang/MambaByte_Stories
Text Generation
•
Updated
•
23
JunxiongWang/MambaByte_Arxiv
Text Generation
•
Updated
•
13
•
3
JunxiongWang/MambaByte_PG19_353M
Text Generation
•
Updated
•
10
JunxiongWang/MambaByte_Books
Text Generation
•
Updated
•
61
•
1
JunxiongWang/MambaByte_Code
Text Generation
•
Updated
•
323
•
2
JunxiongWang/MambaByte_PG19_972M
Text Generation
•
Updated
•
25
JunxiongWang/Mamba2InLlama_1
Updated
•
13
•
1
JunxiongWang/Mamba2InLlama_0_50
Updated
•
318
JunxiongWang/Mamba2InLlama_0_75
Updated
•
15
JunxiongWang/MambaInLlama_0_50
Updated
•
47