Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8" /> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /> | |
| <title>PyTorch × Transformers Journey</title> | |
| <!-- Google Fonts --> | |
| <link href="https://fonts.googleapis.com/css2?family=Inter:wght@300;400;600;800&family=Fira+Code:wght@400;600&display=swap" rel="stylesheet" /> | |
| <!-- Reveal.js core & dark theme base --> | |
| <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/reveal.js@5/dist/reset.css" /> | |
| <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/reveal.js@5/dist/reveal.css" /> | |
| <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/reveal.js@5/dist/theme/black.css" id="theme" /> | |
| <!-- Highlight.js --> | |
| <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/styles/github-dark.min.css" /> | |
| <!-- Animations --> | |
| <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/animate.css@4/animate.min.css" /> | |
| <style> | |
| :root { | |
| --accent-primary: #ee4c2c; /* PyTorch orange‑red */ | |
| --accent-secondary: #ffb347; /* lighter highlight */ | |
| --bg-gradient-start: #1b1b1b; | |
| --bg-gradient-end: #242424; | |
| } | |
| html, body { font-family: 'Inter', sans-serif; } | |
| .reveal .slides { | |
| background: linear-gradient(135deg, var(--bg-gradient-start), var(--bg-gradient-end)); | |
| } | |
| .reveal h1, .reveal h2, .reveal h3 { color: var(--accent-primary); font-weight: 800; letter-spacing: -0.5px; } | |
| .reveal pre code { font-family: 'Fira Code', monospace; font-size: 0.75em; } | |
| .reveal section img, .reveal section svg { border-radius: 1rem; box-shadow: 0 8px 22px rgba(0,0,0,0.4); } | |
| .fragment.highlight-current-blue.visible { color: var(--accent-secondary) ; } | |
| /* slide-density patch */ | |
| .reveal h1 { font-size: 2.6rem; line-height: 1.1; } | |
| .reveal h2 { font-size: 1.9rem; line-height: 1.15; } | |
| .reveal h3 { font-size: 1.4rem; line-height: 1.2; } | |
| .reveal p, .reveal li { font-size: 1.7rem; line-height: 1.35; } | |
| .reveal pre code { font-size: 0.67em; } | |
| /* Make <strong> more vibrant and aligned with the accent */ | |
| .reveal strong { | |
| color: var(--accent-secondary); /* orange highlight */ | |
| font-weight: 800; | |
| } | |
| /* Make <code> more obvious: change background, font, and padding */ | |
| .reveal code { | |
| background: rgba(255, 255, 255, 0.1); | |
| color: #ffd080; | |
| padding: 0.15em 0.4em; | |
| border-radius: 0.3em; | |
| font-family: 'Fira Code', monospace; | |
| font-size: 0.95em; | |
| } | |
| @media (max-width: 1024px) { .reveal h1{font-size:2.2rem;} .reveal h2{font-size:1.6rem;} } | |
| .reveal table td, .reveal table th { font-size: 0.85rem; padding: 4px 8px; } | |
| body::after { | |
| content: ""; | |
| position: fixed; | |
| bottom: 3.5em; | |
| left: 3.5em; | |
| width: 270px; /* desired size */ | |
| height: 117px; | |
| background-image: url(assets/py2.png); | |
| background-size: contain; | |
| background-repeat: no-repeat; | |
| z-index: 9999; | |
| box-shadow: 5px 5px 10px #000; | |
| pointer-events: none; | |
| } | |
| </style> | |
| </head> | |
| <body> | |
| <div class="reveal"> | |
| <div class="slides"> | |
| <section> | |
| <img src="assets/screenpage2.png" alt="Full slide image" | |
| style=" | |
| width:120%; | |
| height:110%; | |
| object-fit:cover; | |
| margin-left:-2.5%; | |
| margin-top:-2.5%; | |
| " /> <!-- 1 · Opening --> | |
| </section> | |
| <section data-auto-animate> | |
| <div style="display: flex; align-items: center; justify-content: center; gap: 1.2rem; margin-bottom: 1rem;" class="animate__animated animate__fadeInDown"> | |
| <img src="assets/transparent_PyTorch.png" alt="PyTorch Logo" style="height: 48px;" /> | |
| <span style="color: white; font-size: 2.4rem; font-weight: 700;">×</span> | |
| <img src="assets/head_logo.svg" alt="Transformers Logo" style="height: 48px;" /> | |
| </div> | |
| <h1 class="animate__animated animate__fadeInDown">PyTorch × Transformers Journey</h1> | |
| <h3 class="animate__animated animate__fadeInDown animate__delay-1s">Pythonicity, Autodiff & Modularity in Modern AI</h3> | |
| <p class="animate__animated animate__fadeInUp animate__delay-2s">Pablo Montalvo‑Leroux · ML Engineer @ Hugging Face</p> | |
| </section> | |
| <section> | |
| <h2>2016‑2018: Backprop & Birth Pangs</h2> | |
| <p>The journey began with uncertainty: back in 2016, machine learning was far from standardized. Tools like Theano and CNTK were fading, and many of us—myself included—were jumping framework to framework. It was a time of raw experimentation.</p> | |
| <ul> | |
| <li>Frameworks were in flux; few stuck around.</li> | |
| <li>MLPs evolved to RNNs and LSTMs.</li> | |
| <li><strong>2017, Attention, then 2018: BERT</strong> arrives, blowing the roof off what's possible.</li> | |
| </ul> | |
| <p class="fragment">But reproducing results remained frustratingly difficult.</p> | |
| </section> | |
| <section> | |
| <h2>Transformers × PyTorch: Reproducibility</h2> | |
| <p>That all changed with <code>pytorch-pretrained-bert</code>, the predecessor to Transformers. Suddenly, the magic of BERT was available in an interface that made sense.</p> | |
| <div style="display: flex; gap: 2rem; justify-content: space-between; margin-top: 2rem;"> | |
| <div style="flex: 1; background: #2d2d2d; padding: 1.2rem; border-radius: 1rem; box-shadow: 0 4px 12px rgba(0,0,0,0.3);"> | |
| <p style="font-weight: 800; color: var(--accent-primary); margin-bottom: 0.6rem;"> | |
| 🧩 Simpler Interface | |
| </p> | |
| <p>No static graphs, just Python functions and PyTorch modules.</p> | |
| </div> | |
| <div style="flex: 1; background: #2d2d2d; padding: 1.2rem; border-radius: 1rem; box-shadow: 0 4px 12px rgba(0,0,0,0.3);"> | |
| <p style="font-weight: 800; color: var(--accent-primary); margin-bottom: 0.6rem;"> | |
| ✨ Hackability | |
| </p> | |
| <p>Readable, hackable code meant results could be shared, reproduced, improved.</p> | |
| </div> | |
| <div style="flex: 1; background: #2d2d2d; padding: 1.2rem; border-radius: 1rem; box-shadow: 0 4px 12px rgba(0,0,0,0.3);"> | |
| <p style="font-weight: 800; color: var(--accent-primary); margin-bottom: 0.6rem;"> | |
| 🚀 Community Shift | |
| </p> | |
| <p>This shifted the research community towards PyTorch.</p> | |
| </div> | |
| </div> | |
| </section> | |
| <!-- 3 · Static vs Dynamic Graphs --> | |
| <section> | |
| <h2>Static vs Dynamic Graphs</h2> | |
| <p>Static graphs require you to compile, wait, and cross fingers the bug reproduces.</p> | |
| <p>Dynamic graphs mean you can drop <code>pdb.set_trace()</code> anywhere and continue iterating.</p> | |
| <p>Nowadays <code>torch.compile</code> gives the best of both worlds: write dynamically, ship something ahead‑of‑time optimised.</p> | |
| </section> | |
| <!-- 4 · Dynamic Graphs Enabled Contribution --> | |
| <section> | |
| <h2>Dynamic Graphs Enabled Contribution</h2> | |
| <ul> | |
| <li>Developers debug at line‑rate — no cold‑start recompiles.</li> | |
| <li>Pull‑requests remained reproducible overnight, which accelerated trust.</li> | |
| <li>Static‑graph alternatives stalled and the community consolidated around PyTorch.</li> | |
| </ul> | |
| </section> | |
| <section> | |
| <h2>Clone the Paper Tonight → Tweak Tomorrow</h2> | |
| <p>PyTorch lowered the barrier to implementation — Transformers built on top of that simplicity.</p> | |
| <div style="display: flex; gap: 1.5rem; margin-top: 2rem;"> | |
| <div style="flex: 1; background: #2d2d2d; padding: 1.2rem; border-radius: 1rem; box-shadow: 0 4px 12px rgba(0,0,0,0.3);"> | |
| <p style="font-weight: 800; color: var(--accent-primary); margin-bottom: 0.5rem;">🔍 Live Debugging</p> | |
| <p>2018: BERT fine-tunes meant <code>print(tensor)</code>, not <em>recompile & hope</em>.</p> | |
| </div> | |
| <div style="flex: 1; background: #2d2d2d; padding: 1.2rem; border-radius: 1rem; box-shadow: 0 4px 12px rgba(0,0,0,0.3);"> | |
| <p style="font-weight: 800; color: var(--accent-primary); margin-bottom: 0.5rem;">🤝 Fast Review</p> | |
| <p>Patches were understandable and reproducible — merged quickly, verified quickly.</p> | |
| </div> | |
| <div style="flex: 1; background: #2d2d2d; padding: 1.2rem; border-radius: 1rem; box-shadow: 0 4px 12px rgba(0,0,0,0.3);"> | |
| <p style="font-weight: 800; color: var(--accent-primary); margin-bottom: 0.5rem;">⚡ Fast Iteration</p> | |
| <p>Experiments shifted from <em>weeks</em> to <strong>hours</strong> — feedback cycles accelerated.</p> | |
| </div> | |
| </div> | |
| </section> | |
| <!-- 6 · One Model · One File --> | |
| <section> | |
| <h2>“One Model · One File” — Why it Matters</h2> | |
| <pre><code class="language-python" data-trim data-noescape> | |
| # modeling_bert.py — single source of truth | |
| class BertConfig(PretrainedConfig): | |
| ... | |
| class BertSelfAttention(nn.Module): | |
| ... | |
| class BertLayer(nn.Module): | |
| ... | |
| class BertModel(PreTrainedModel): | |
| def __init__(self, config): | |
| super().__init__(config) | |
| self.embeddings = BertEmbeddings(config) | |
| self.encoder = nn.ModuleList( | |
| [BertLayer(config) for _ in range(config.num_hidden_layers)] | |
| ) | |
| self.init_weights() | |
| </code></pre> | |
| <ul> | |
| <li>All layers, forward pass, and <code>from_pretrained()</code> logic live together.</li> | |
| <li>No cross‑file inheritance maze — copy to Colab, hack, and run.</li> | |
| <li>Reviewers diff one file; merge time dropped from days to hours.</li> | |
| </ul> | |
| </section> | |
| <!-- 8 · Paradigms come at a cost --> | |
| <section> | |
| <h2>Paradigms Come at a Cost</h2> | |
| <div style="display: grid; grid-template-columns: repeat(2, 1fr); gap: 1.5rem; margin-top: 2rem;"> | |
| <div style="background: #2d2d2d; padding: 1.2rem; border-radius: 1rem; box-shadow: 0 4px 12px rgba(0,0,0,0.3);"> | |
| <p style="font-weight: 800; color: var(--accent-primary); margin-bottom: 0.5rem;">📈 Community Growth</p> | |
| <p>The scientific and engineering ML community thrived with Transformers.</p> | |
| </div> | |
| <div style="background: #2d2d2d; padding: 1.2rem; border-radius: 1rem; box-shadow: 0 4px 12px rgba(0,0,0,0.3);"> | |
| <p style="font-weight: 800; color: var(--accent-primary); margin-bottom: 0.5rem;">🔥 PyTorch Synergy</p> | |
| <p>Transformers and PyTorch grew together — adoption fed back into both ecosystems.</p> | |
| </div> | |
| <div style="background: #2d2d2d; padding: 1.2rem; border-radius: 1rem; box-shadow: 0 4px 12px rgba(0,0,0,0.3);"> | |
| <p style="font-weight: 800; color: var(--accent-primary); margin-bottom: 0.5rem;">🛠️ Maintenance Pressure</p> | |
| <p>We duplicate code on purpose — to preserve clarity, portability, and hackability.</p> | |
| </div> | |
| <div class="fragment" style="background: #2d2d2d; padding: 1.2rem; border-radius: 1rem; box-shadow: 0 4px 12px rgba(0,0,0,0.3);"> | |
| <p style="font-weight: 800; color: var(--accent-primary); margin-bottom: 0.5rem;">🧬 Pythonic Modularity</p> | |
| <p>The <strong>Modularity</strong> of python is never far :)</p> | |
| </div> | |
| </div> | |
| </section> | |
| <!-- 8 · Back to Python: Mary Shelley Mode --> | |
| <section> | |
| <h2>Back to Python: Modular “Mary Shelley” Mode</h2> | |
| <p>Compose new blocks via subclass & override.</p> | |
| <pre><code class="language-python" data-trim> | |
| class GlmMLP(Phi3MLP): | |
| pass | |
| class GlmAttention(LlamaAttention): | |
| def __init__(self, config, layer_idx=None): | |
| super().__init__(config, layer_idx) | |
| self.o_proj = nn.Linear(config.num_attention_heads * self.head_dim, | |
| config.hidden_size, bias=False) | |
| class GlmForCausalLM(LlamaForCausalLM): | |
| pass | |
| </code></pre> | |
| <p>AST expands → full modeling file, still hackable.</p> | |
| </section> | |
| <section> | |
| <h2>Back to Python: It's alive!</h2> | |
| <p>All the code becomes runnable and a self-contained model definition</p> | |
| <pre><code class="language-python" data-trim> | |
| class GlmMLP(nn.Module): | |
| def __init__(self, config): | |
| super().__init__() | |
| self.config = config | |
| self.gate_up_proj = nn.Linear(config.hidden_size, 2 * config.intermediate_size, bias=False) | |
| self.down_proj = nn.Linear(config.intermediate_size, config.hidden_size, bias=False) | |
| self.activation_fn = ACT2FN[config.hidden_act] | |
| def forward(self, hidden_states: torch.FloatTensor) -> torch.FloatTensor: | |
| up_states = self.gate_up_proj(hidden_states) | |
| gate, up_states = up_states.chunk(2, dim=-1) | |
| up_states = up_states * self.activation_fn(gate) | |
| return self.down_proj(up_states) | |
| class GlmAttention(nn.Module): | |
| """Multi-headed attention from 'Attention Is All You Need' paper""" | |
| def __init__(self, config: GlmConfig, layer_idx: Optional[int] = None): | |
| super().__init__() | |
| self.config = config | |
| self.layer_idx = layer_idx | |
| self.head_dim = getattr(config, "head_dim", config.hidden_size // config.num_attention_heads) | |
| self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads | |
| self.scaling = self.head_dim**-0.5 | |
| self.attention_dropout = config.attention_dropout | |
| self.is_causal = True | |
| self.q_proj = nn.Linear( | |
| config.hidden_size, config.num_attention_heads * self.head_dim, bias=config.attention_bias | |
| ) | |
| self.k_proj = nn.Linear( | |
| config.hidden_size, config.num_key_value_heads * self.head_dim, bias=config.attention_bias | |
| ) | |
| self.v_proj = nn.Linear( | |
| config.hidden_size, config.num_key_value_heads * self.head_dim, bias=config.attention_bias | |
| ) | |
| self.o_proj = nn.Linear(config.num_attention_heads * self.head_dim, config.hidden_size, bias=False) | |
| def forward( | |
| self, | |
| hidden_states: torch.Tensor, | |
| position_embeddings: Tuple[torch.Tensor, torch.Tensor], | |
| attention_mask: Optional[torch.Tensor], | |
| past_key_value: Optional[Cache] = None, | |
| cache_position: Optional[torch.LongTensor] = None, | |
| **kwargs: Unpack[FlashAttentionKwargs], | |
| ) -> Tuple[torch.Tensor, Optional[torch.Tensor], Optional[Tuple[torch.Tensor]]]: | |
| input_shape = hidden_states.shape[:-1] | |
| hidden_shape = (*input_shape, -1, self.head_dim) | |
| query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2) | |
| key_states = self.k_proj(hidden_states).view(hidden_shape).transpose(1, 2) | |
| value_states = self.v_proj(hidden_states).view(hidden_shape).transpose(1, 2) | |
| cos, sin = position_embeddings | |
| query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin) | |
| if past_key_value is not None: | |
| # sin and cos are specific to RoPE models; cache_position needed for the static cache | |
| cache_kwargs = {"sin": sin, "cos": cos, "cache_position": cache_position} | |
| key_states, value_states = past_key_value.update(key_states, value_states, self.layer_idx, cache_kwargs) | |
| attention_interface: Callable = eager_attention_forward | |
| if self.config._attn_implementation != "eager": | |
| if self.config._attn_implementation == "sdpa" and kwargs.get("output_attentions", False): | |
| logger.warning_once( | |
| "`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to " | |
| 'eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.' | |
| ) | |
| else: | |
| attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation] | |
| attn_output, attn_weights = attention_interface( | |
| self, | |
| query_states, | |
| key_states, | |
| value_states, | |
| attention_mask, | |
| dropout=0.0 if not self.training else self.attention_dropout, | |
| scaling=self.scaling, | |
| **kwargs, | |
| ) | |
| attn_output = attn_output.reshape(*input_shape, -1).contiguous() | |
| attn_output = self.o_proj(attn_output) | |
| return attn_output, attn_weights | |
| @use_kernel_forward_from_hub("RMSNorm") | |
| class GlmRMSNorm(nn.Module): | |
| def __init__(self, hidden_size, eps=1e-6): | |
| """ | |
| GlmRMSNorm is equivalent to T5LayerNorm | |
| """ | |
| super().__init__() | |
| self.weight = nn.Parameter(torch.ones(hidden_size)) | |
| self.variance_epsilon = eps | |
| def forward(self, hidden_states): | |
| input_dtype = hidden_states.dtype | |
| hidden_states = hidden_states.to(torch.float32) | |
| variance = hidden_states.pow(2).mean(-1, keepdim=True) | |
| hidden_states = hidden_states * torch.rsqrt(variance + self.variance_epsilon) | |
| return self.weight * hidden_states.to(input_dtype) | |
| def extra_repr(self): | |
| return f"{tuple(self.weight.shape)}, eps={self.variance_epsilon}" | |
| class GlmRotaryEmbedding(nn.Module): | |
| def __init__(self, config: GlmConfig, device=None): | |
| super().__init__() | |
| # BC: "rope_type" was originally "type" | |
| if hasattr(config, "rope_scaling") and config.rope_scaling is not None: | |
| self.rope_type = config.rope_scaling.get("rope_type", config.rope_scaling.get("type")) | |
| else: | |
| self.rope_type = "default" | |
| self.max_seq_len_cached = config.max_position_embeddings | |
| self.original_max_seq_len = config.max_position_embeddings | |
| self.config = config | |
| self.rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type] | |
| inv_freq, self.attention_scaling = self.rope_init_fn(self.config, device) | |
| self.register_buffer("inv_freq", inv_freq, persistent=False) | |
| self.original_inv_freq = self.inv_freq | |
| </code></pre> | |
| <p> We keep hackability while reconnecting with Python working paradigms.</p> | |
| </section> | |
| <!-- 9 · Logit Debugger --> | |
| <section> | |
| <h2>Logit Debugger: Trust but Verify</h2> | |
| <ul> | |
| <li>Hook every <code>nn.Module</code>; dump logits layer‑by‑layer</li> | |
| <li>Spot ε‑level drifts (LayerNorm, FP16 underflow…)</li> | |
| <li>JSON traces diffable in CI</li> | |
| <img data-src="assets/visual_debugger.png" alt="Visual debugger" /> | |
| </ul> | |
| </section> | |
| <!-- 10 · DTensor & TP API --> | |
| <section> | |
| <h2>DTensor & Tensor‑Parallel API</h2> | |
| <p>Before, changing to Tensor Parallel meant changing the code.</p> | |
| <pre><code class="language-python" data-trim data-noescape> | |
| from transformers.modeling_utils import PreTrainedModel | |
| from megatron.model import ColumnParallelLinear, RowParallelLinear | |
| class MyTPModel(PreTrainedModel): | |
| def __init__(self, config): | |
| super().__init__(config) | |
| self.q_proj = ColumnParallelLinear(config.hidden_size, config.hidden_size) | |
| self.k_proj = ColumnParallelLinear(config.hidden_size, config.hidden_size) | |
| self.v_proj = ColumnParallelLinear(config.hidden_size, config.hidden_size) | |
| self.o_proj = RowParallelLinear(config.hidden_size, config.hidden_size) | |
| </code></pre> | |
| </section> | |
| <!-- 11 · Zero‑Config Parallelism --> | |
| <section> | |
| <h2>Zero‑Config Tensor Parallelism</h2> | |
| <p>The <code>tp_plan</code> JSON keeps model code pristine and declarative.</p> | |
| <pre><code class="language-json" data-trim data-noescape>{ | |
| "layer.*.self_attn.q_proj": "colwise", | |
| "layer.*.self_attn.k_proj": "colwise", | |
| "layer.*.self_attn.v_proj": "colwise", | |
| "layer.*.self_attn.o_proj": "rowwise" | |
| }</code></pre> | |
| <p>Translated to</p> | |
| <pre><code class="language-python" data-trim data-noescape> | |
| def translate_to_torch_parallel_style(style: str): | |
| if style == "colwise": | |
| return ColwiseParallel() | |
| elif style == "rowwise": | |
| return RowwiseParallel() | |
| # … | |
| </code></pre> | |
| <p>One JSON → 100 B param model on 8 GPUs. Change the plan, not the code.</p> | |
| </section> | |
| <!-- 12 · Cache Allocator --> | |
| <section> | |
| <h2>Improvements, Load faster & stronger: Cache Allocator</h2> | |
| <p>0‑copy weight sharding, single cuda Malloc</p> | |
| <p>Faster model loads, even for a 50-shards 100B model (when we were sprinting Llama4!)</p> | |
| <img data-src="assets/fastload.png" alt="SurprisedLewis" /> | |
| </section> | |
| <!-- 15 · Why Python wins --> | |
| <section> | |
| <h2>Why Python Wins</h2> | |
| <ul> | |
| <li>Low entry barrier (although hard to master)</li> | |
| <li>High‑level semantics express low‑level intent</li> | |
| <li>Seamless C++/Rust extension points</li> | |
| </ul> | |
| </section> | |
| <!-- 16 · Where Python can bite --> | |
| <section> | |
| <h2>Where Python can bite 🐍</h2> | |
| <ul> | |
| <li>Interpreter overhead on microkernels (token‑by‑token decode)</li> | |
| <li>GIL can throttle async host‑side work</li> | |
| <li>Easy to under‑optimise code fresh out of the lab</li> | |
| </ul> | |
| <p class="fragment">All of these can be mitigated: Triton, compiled custom ops, compile‑time fallback, <strong>custom kernels</strong></p> | |
| </section> | |
| <!-- 17 · Kernel Hub --> | |
| <section> | |
| <h2>Kernel Hub: Optimised Ops from the Community</h2> | |
| <p>Kernel Hub lets any Python program <em>download and hot‑load</em> compiled CUDA/C++ kernels directly from the Hugging Face Hub at runtime.</p> | |
| <ul> | |
| <li><strong>Portable</strong> – kernels work from arbitrary paths outside <code>PYTHONPATH</code>.</li> | |
| <li><strong>Unique</strong> – load multiple versions of the same op side‑by‑side in one process.</li> | |
| <li><strong>Compatible</strong> – every kernel targets all recent PyTorch wheels (CUDA, ROCm, CPU) and C‑library ABIs.</li> | |
| </ul> | |
| <pre><code class="language-python" data-trim data-noescape> | |
| import torch | |
| from kernels import get_kernel | |
| # Download optimised kernels from the Hugging Face Hub | |
| activation = get_kernel("kernels-community/activation") | |
| x = torch.randn(10, 10, dtype=torch.float16, device="cuda") | |
| y = torch.empty_like(x) | |
| activation.gelu_fast(y, x) | |
| print(y) | |
| </code></pre> | |
| <p>Same Transformer code — now with a <strong>3× faster</strong> GELU on A100s.</p> | |
| </section> | |
| <section> | |
| <h2>API Design Lessons</h2> | |
| <div style="display: flex; gap: 1.2rem; margin-top: 1.2rem;"> | |
| <div style="flex: 1; background: #2c2c2c; padding: 0.9rem; border-radius: 0.6rem; box-shadow: 0 3px 10px rgba(0,0,0,0.25); font-size: 1.35rem;"> | |
| <p style="font-weight: 700; color: var(--accent-primary); margin-bottom: 0.4rem;">🔍 Make Easy Things Obvious</p> | |
| <p style="margin-bottom: 0.4rem;">Models load in <code>one line</code> — no boilerplate.</p> | |
| <pre><code class="language-python" style="font-size: 0.75em;">model = AutoModel.from_pretrained("bert-base-uncased")</code></pre> | |
| </div> | |
| <div style="flex: 1; background: #2c2c2c; padding: 0.9rem; border-radius: 0.6rem; box-shadow: 0 3px 10px rgba(0,0,0,0.25); font-size: 1.35rem;"> | |
| <p style="font-weight: 700; color: var(--accent-primary); margin-bottom: 0.4rem;">📄 Paper-to-Repo Diff ≈ 0</p> | |
| <p style="margin-bottom: 0.4rem;">Code reflects architecture directly.</p> | |
| <pre><code class="language-python" style="font-size: 0.75em;">class LlamaAttention(nn.Module): ...</code></pre> | |
| </div> | |
| </div> | |
| <div style="display: flex; gap: 1.2rem; margin-top: 1.2rem;"> | |
| <div style="flex: 1; background: #2c2c2c; padding: 0.9rem; border-radius: 0.6rem; box-shadow: 0 3px 10px rgba(0,0,0,0.25); font-size: 1.35rem;"> | |
| <p style="font-weight: 700; color: var(--accent-primary); margin-bottom: 0.4rem;">🚀 Prototyping → Production</p> | |
| <p style="margin-bottom: 0.4rem;">Same model runs in vLLM for deployment:</p> | |
| <pre><code class="language-python" style="font-size: 0.75em;">LLM(model="llama", model_impl="transformers")</code></pre> | |
| </div> | |
| <div style="flex: 1; background: #2c2c2c; padding: 0.9rem; border-radius: 0.6rem; box-shadow: 0 3px 10px rgba(0,0,0,0.25); font-size: 1.35rem;"> | |
| <p style="font-weight: 700; color: var(--accent-primary); margin-bottom: 0.4rem;">🎛️ Hide Sharding, Show Intent</p> | |
| <p style="margin-bottom: 0.4rem;">Declarative TP via config:</p> | |
| <pre><code class="language-json" style="font-size: 0.75em;">"q_proj": "colwise"</code></pre> | |
| </div> | |
| </div> | |
| <p style="font-size: 1.35rem; margin-top: 1.6rem;"> | |
| We tune radios without building RF amps. ML should feel the same. | |
| </p> | |
| <p class="fragment" style="font-size: 1.35rem;"> | |
| …while empowering those who do build the amps. | |
| </p> | |
| </section> | |
| <!-- 14 · Rise of Multimodality --> | |
| <section> | |
| <h2>Rise of Multimodality</h2> | |
| <pre><code class="language-python" data-trim data-noescape> | |
| processor = AutoProcessor.from_pretrained("Qwen/Qwen3-8B") | |
| model = AutoModelForConditionalGeneration.from_pretrained("Qwen/Qwen3-8B") | |
| </code></pre> | |
| <p>Same API across text · vision · audio</p> | |
| <p>More and more models, with specific processing - need to uniformize</p> | |
| </section> | |
| <section> | |
| <h2>Rise of Multimodality: torch-powered processing</h2> | |
| <p>Torch and torchvision ops have replaced np + PIL defaults in transformers</p> | |
| <img data-src="assets/normalize_time_torch.webp" width="80%" height="600" alt="Fast load" /> | |
| </section> | |
| <!-- 19 · Model Growth by Modality --> | |
| <section> | |
| <h2>Model Growth by Modality</h2> | |
| <embed style="border: none;" src="assets/plot_tryout.html" dpi="300" width="70%" height="500px" /> | |
| </section> | |
| <section> | |
| <h2>Beyond Transformers: Ecosystem Reuse</h2> | |
| <p><strong>Transformers</strong> makes modeling easy. <strong>vLLM</strong> makes inference fast.</p> | |
| <div style="display: flex; gap: 2rem; margin-top: 2rem;"> | |
| <div style="flex: 1;"> | |
| <p><strong>🔧 Prototype with Transformers:</strong></p> | |
| <pre><code class="language-python" data-trim data-noescape> | |
| from transformers import pipeline | |
| pipe = pipeline("text-generation", model="meta-llama/Llama-3.2-1B") | |
| print(pipe("The future of AI is")[0]["generated_text"]) | |
| </code></pre> | |
| </div> | |
| <div style="flex: 1;"> | |
| <img src="assets/vLLM-Full-Logo.png" alt="vLLM Illustration" style="border-radius: 1rem; box-shadow: 0 0 12px #000; width: 100%;" /> | |
| </div> | |
| </div> | |
| </section> | |
| <section> | |
| <h2>Deploy with vLLM — No Rewrite Needed</h2> | |
| <p><strong>vLLM</strong> supports <code>transformers</code> models out of the box. </p> | |
| <p>Just specify <code>model_impl="transformers"</code> if needed:</p> | |
| <pre><code class="language-python" data-trim data-noescape> | |
| from vllm import LLM, SamplingParams | |
| llm = LLM(model="meta-llama/Llama-3.2-1B", model_impl="transformers") | |
| params = SamplingParams(max_tokens=20) | |
| outputs = llm.generate("The future of AI is", sampling_params=params) | |
| print(outputs[0].outputs[0].text) | |
| </code></pre> | |
| <p class="fragment">We also support SGLang now, along with thousands of other libraries! </p> | |
| </section> | |
| <section> | |
| <h2 style="margin-bottom: 1rem;"> | |
| Transformers × PyTorch — Enabling the Community | |
| </h2> | |
| <img src="assets/transformers_as_ref.png" alt="Transformers as Reference" | |
| style=" | |
| width: 120%; | |
| height: 110%; | |
| object-fit: cover; | |
| margin-left: -2.5%; | |
| margin-top: -2.5%; | |
| " /> | |
| </section> | |
| <section> | |
| <h2>Takeaways & The Future</h2> | |
| <div style="display: grid; grid-template-columns: repeat(2, 1fr); gap: 1rem; margin-top: 1.5rem;"> | |
| <div style="background: #2d2d2d; padding: 1rem; border-radius: 0.8rem;"> | |
| <p style="font-weight: 700; font-size: 1.4rem; color: var(--accent-primary); margin-bottom: 0.4rem;"> | |
| 🤝 Symbiotic Growth | |
| </p> | |
| <p style="display: flex; align-items: center; gap: 0.4rem; font-size: 1.4rem;"> | |
| <img src="assets/transparent_PyTorch.png" alt="PyTorch" style="height: 1.4rem;" /> | |
| <code> PyTorch</code> & <code>transformers</code> grow together | |
| <img src="assets/head_logo.svg" alt="Transformers" style="height: 1.4rem;" /> | |
| </p> | |
| </div> | |
| <div style="background: #2d2d2d; padding: 1rem; border-radius: 0.8rem;"> | |
| <p style="font-weight: 700; font-size: 1.4rem; color: var(--accent-primary); margin-bottom: 0.4rem;"> | |
| 🧠 Pythonicity × Pragmatism | |
| </p> | |
| <p style="font-size: 1.4rem;">High-level code, low-level control — a winning combination for fast iteration.</p> | |
| </div> | |
| <div style="background: #2d2d2d; padding: 1rem; border-radius: 0.8rem;"> | |
| <p style="font-weight: 700; font-size: 1.4rem; color: var(--accent-primary); margin-bottom: 0.4rem;"> | |
| 🚢 Models Ship Faster | |
| </p> | |
| <p style="font-size: 1.4rem;">Open-source models are scaling up — and landing in users' hands faster than ever.</p> | |
| </div> | |
| <div style="background: #2d2d2d; padding: 1rem; border-radius: 0.8rem;"> | |
| <p style="font-weight: 700; font-size: 1.4rem; color: var(--accent-primary); margin-bottom: 0.4rem;"> | |
| 📚 Source of Truth for Model Definitions | |
| </p> | |
| <p style="font-size: 1.4rem;">We aim to be the canonical reference — while enabling the community to build, remix, and deploy at scale.</p> | |
| </div> | |
| </div> | |
| <p style="margin-top: 1.5rem; font-size: 1.3rem;"> | |
| <a href="https://huggingface.co/transformers/contribute" target="_blank"> | |
| hf.co/transformers/contribute | |
| </a> | |
| </p> | |
| </section> | |
| </div> | |
| </div> | |
| <!-- Reveal.js core --> | |
| <script src="https://cdn.jsdelivr.net/npm/reveal.js@5/dist/reveal.js"></script> | |
| <script src="https://cdn.jsdelivr.net/npm/reveal.js@5/plugin/highlight/highlight.js"></script> | |
| <script src="https://cdn.jsdelivr.net/npm/reveal.js@5/plugin/notes/notes.js"></script> | |
| <!-- Plotly for interactive charts --> | |
| <script src="https://cdn.plot.ly/plotly-2.31.1.min.js"></script> | |
| <script> | |
| Reveal.initialize({ | |
| hash: true, | |
| slideNumber: true, | |
| transition: 'slide', | |
| backgroundTransition: 'convex', | |
| plugins: [ RevealHighlight, RevealNotes ] | |
| }); | |
| </script> | |
| </body> | |
| </html> | |