event

GPT-2 Visualization Release

4 items · started 2026-10-27 · closed 2026-11-01

  1. Compositional generalization remains a persistent limitation of transformer language models: even when a model learns to combine known facts correctly in-distribution (ID), the same compositional mechanism may fail when applied to out-of-d…

  2. Pruned language models are judged almost entirely by accuracy and perplexity, which assumes that preserved output behaviour implies preserved internal computation. We test this assumption with a factorial pruning study on GPT-2, crossing t…

  3. Knowledge distillation compresses large language models into compact students that preserve aggregate metrics, yet the mechanistic consequences of depth-reduction distillation remain largely uncharacterized. This work applies linear CKA, l…

  4. An interactive 3D walk through GPT-2. Type a sentence and watch every matrix operation, with the real numbers, computed in your browser.

← all threads