Ideas

Deep Learning

Deep learning notes and projects covering neural networks, sequence models, language models, generative modeling, and interpretability.

Neural networks, mostly the language side of them. There's writing on search and semantic search, how language models work, a survey of text diffusion, and some interpretability work that looks at what a model has actually learned.

It overlaps a lot with the machine learning page. This is the part that's specifically about deep learning and language models.

deep-learningNLPgenerative-models
Thread Retro 12 parts Updated
Abstract shapes
Retro Complete 8 min read

Transformer Circuits Reading Notes

Paper-by-paper notes on Anthropic's Transformer Circuits thread: core claims, methods in plain language, and what the evidence does and doesn't show.
Abstract shapes
Note Complete 9 min read

Notes on A Mathematical Framework for Transformer Circuits

Notes on the 2021 paper that reframes transformers as operations on a shared residual stream: QK/OV circuits, virtual weights, and path decompositions of attention-only models.
Abstract shapes
Note Complete 8 min read

Notes on In-Context Learning and Induction Heads

Notes on induction heads, the training phase change in which they form, and the six-part argument connecting them to in-context learning.
Abstract shapes
Note Complete 11 min read

Notes on Toy Models of Superposition

Notes on how sparse features fit into superposition, the geometry and interference that result, and a toy-model account of memorization and double descent.
Abstract shapes
Note Complete 13 min read

Notes on Towards Monosemanticity and Scaling Monosemanticity

Notes on decomposing MLP activations into interpretable features with sparse autoencoders, from a one-layer transformer to Claude 3 Sonnet.
Abstract stacks
Note Complete 9 min read

Notes on Sparse Crosscoders

Notes on sparse crosscoders: one shared feature dictionary across layers or across models, and what that enables for cross-layer analysis and model diffing.
Abstract biology
Note Complete 18 min read

Notes on Circuit Tracing and the Biology of a Large Language Model

Notes on cross-layer transcoders, local replacement models, and attribution graphs — and the case-study companion paper on Claude 3.5 Haiku.
Abstract biology
Note Complete 16 min read

Notes on Tracing Attention Computation Through Feature Interactions

Notes on QK attributions, attention-head loadings, and what feature interactions reveal about why a transformer attends to one token rather than another.
Two curved bands of colored circles and squares joined by fine lines, collaged over botanical illustrations and paper textures
Note Complete 23 min read

Notes on When Models Manipulate Manifolds

Notes on the geometry of a counting task: how Claude 3.5 Haiku tracks character position on a curved feature manifold, and what attention heads do to that curve.
A pale botanical specimen in profile, with a small coral pollen-like signal entering its lower inner structure. Inside a circular glass inspection window, an inner ring of petals has already shifted toward it, while the outer blossom remains unchanged. Ghosted nearby studies remain unresponsive.
Note Complete 22 min read

Notes on Emergent Introspective Awareness

Notes on concept injection: how you test whether a language model's claims about its own thoughts are grounded in its actual internal state, and what the results do and do not show.
Three botanical specimens in blue, green and coral, their stems drawn into a narrow vertical stack of overlapping colored bars and emerging on the far side as three fanned sprays of the same colors, much enlarged
Note Complete 13 min read

Notes on Emotion Concepts and their Function

Notes on emotion vectors in Claude Sonnet 4.5: what they represent, why "just character simulation" is not the dismissal it sounds like, and what steering them does to blackmail and reward hacking rates.
Botanical illustrations collaged on either side of a row of translucent stacked planes, with light converging through two crystal prisms at the center and fanning out again
Note Complete 22 min read

Notes on Natural Language Autoencoders

Notes on NLAs as expressive hypothesis generators: why their activation descriptions stay readable, when those descriptions help an investigation, and why the prose still needs independent checks.
Botanical illustrations collaged on either side of a row of translucent stacked planes, with light converging through two crystal prisms at the center and fanning out again
Note Complete 29 min read

Notes on Verbalizable Representations Form a Global Workspace (J-space)

Notes on the Jacobian lens, evidence that token-indexed verbalizable directions mediate report and some reasoning, and the limits of treating the J-space as a global workspace.