Mamba distilled from Zephyr. The Mamba in the Llama: Distilling and Accelerating Hybrid Models (https://arxiv.org/abs/2408.15237).
Junxiong Wang
JunxiongWang
AI & ML interests
Attention Free Model / Subquadratic Language Models
Organizations
Collections
4
models
28
JunxiongWang/MambaByte_Stories
Text Generation
•
Updated
•
19
JunxiongWang/MambaByte_Arxiv
Text Generation
•
Updated
•
15
•
3
JunxiongWang/MambaByte_PG19_353M
Text Generation
•
Updated
•
8
JunxiongWang/MambaByte_Books
Text Generation
•
Updated
•
21
•
1
JunxiongWang/MambaByte_Code
Text Generation
•
Updated
•
477
•
2
JunxiongWang/MambaByte_PG19_972M
Text Generation
•
Updated
•
17
JunxiongWang/Mamba2InLlama_1
Updated
•
8
•
1
JunxiongWang/Mamba2InLlama_0_50
Updated
•
296
JunxiongWang/Mamba2InLlama_0_75
Updated
•
6
JunxiongWang/MambaInLlama_0_50
Updated
•
46