Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
Paper
•
2404.02258
•
Published
•
104
LLM foundations
-
-
Textbooks Are All You Need
Paper • 2306.11644 • Published • 142 -
Jamba: A Hybrid Transformer-Mamba Language Model
Paper • 2403.19887 • Published • 104
Large Language Models Struggle to Learn Long-Tail Knowledge
Paper • 2211.08411 • Published • 3Note This paper shows that LLMs should not be used as libraries! They are not knowledge engines, but rather (feeble) reasoning engines.
-
Language Models are Few-Shot Learners
Paper • 2005.14165 • Published • 11 -
Large Language Models are Zero-Shot Reasoners
Paper • 2205.11916 • Published • 1