event

GPT 2

3 items · started 2026-10-02 · closed 2026-10-04

  1. Why didn't we get GPT-2 in 2005? The ancient Romans were never great at building ships and never tried to explore the Atlantic.

  2. Should low-precision transformer inference use stochastic rounding (SR) or round-to-nearest (RN)? The answer depends on where in the network you look.

  3. Attention-head ablation, zeroing a head and measuring the resulting change in task performance, is a common method for inferring which components of a language model are causally responsible for a behavior. We show using GPT-2 small that t…

← all threads