Transformer Circuits Reading Notes
Paper-by-paper notes on Anthropic's Transformer Circuits thread: core claims, methods in plain language, and what the evidence does and doesn't show.
Notes on A Mathematical Framework for Transformer Circuits
Notes on the 2021 paper that reframes transformers as operations on a shared residual stream: QK/OV circuits, virtual weights, and path decompositions of attention-only models.
Notes on In-Context Learning and Induction Heads
Notes on induction heads, the training phase change in which they form, and the six-part argument connecting them to in-context learning.
Notes on Toy Models of Superposition
Notes on how sparse features fit into superposition, the geometry and interference that result, and a toy-model account of memorization and double descent.
Notes on Towards Monosemanticity and Scaling Monosemanticity
Notes on decomposing MLP activations into interpretable features with sparse autoencoders, from a one-layer transformer to Claude 3 Sonnet.
Notes on Sparse Crosscoders
Notes on sparse crosscoders: one shared feature dictionary across layers or across models, and what that enables for cross-layer analysis and model diffing.
Notes on Circuit Tracing and the Biology of a Large Language Model
Notes on cross-layer transcoders, local replacement models, and attribution graphs — and the case-study companion paper on Claude 3.5 Haiku.
Notes on Tracing Attention Computation Through Feature Interactions
Notes on QK attributions, attention-head loadings, and what feature interactions reveal about why a transformer attends to one token rather than another.
Notes on When Models Manipulate Manifolds
Notes on the geometry of a counting task: how Claude 3.5 Haiku tracks character position on a curved feature manifold, and what attention heads do to that curve.
Notes on Emergent Introspective Awareness
Notes on concept injection: how you test whether a language model's claims about its own thoughts are grounded in its actual internal state, and what the results do and do not show.
Notes on Emotion Concepts and their Function
Notes on emotion vectors in Claude Sonnet 4.5: what they represent, why "just character simulation" is not the dismissal it sounds like, and what steering them does to blackmail and reward hacking rates.
Notes on Natural Language Autoencoders
Notes on NLAs as expressive hypothesis generators: why their activation descriptions stay readable, when those descriptions help an investigation, and why the prose still needs independent checks.
Notes on Verbalizable Representations Form a Global Workspace (J-space)
Notes on the Jacobian lens, evidence that token-indexed verbalizable directions mediate report and some reasoning, and the limits of treating the J-space as a global workspace.
