Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
Paper
•
2404.02258
•
Published
•
106
LLM foundations
-
-
Textbooks Are All You Need
Paper • 2306.11644 • Published • 144 -
Jamba: A Hybrid Transformer-Mamba Language Model
Paper • 2403.19887 • Published • 111
Large Language Models Struggle to Learn Long-Tail Knowledge
Paper • 2211.08411 • Published • 3Note This paper shows that LLMs should not be used as libraries! They are not knowledge engines, but rather (feeble) reasoning engines.
-
Language Models are Few-Shot Learners
Paper • 2005.14165 • Published • 13 -
Large Language Models are Zero-Shot Reasoners
Paper • 2205.11916 • Published • 1 -
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
Paper • 2411.04905 • Published • 125