Research notes and projects on AI safety, evaluations, goal drift, and mechanistic interpretability of model behavior.
Paper-by-paper notes on Anthropic's Transformer Circuits thread: core claims, methods in plain language, and what the evidence does and doesn't show.
Notes on the 2021 paper that reframes transformers as operations on a shared residual stream: QK/OV circuits, virtual weights, and path decompositions of attention-only models.
Notes on induction heads, the training phase change in which they form, and the six-part argument connecting them to in-context learning.
Notes on how sparse features fit into superposition, the geometry and interference that result, and a toy-model account of memorization and double descent.
Notes on decomposing MLP activations into interpretable features with sparse autoencoders, from a one-layer transformer to Claude 3 Sonnet.
Notes on sparse crosscoders: one shared feature dictionary across layers or across models, and what that enables for cross-layer analysis and model diffing.
Notes on cross-layer transcoders, local replacement models, and attribution graphs — and the case-study companion paper on Claude 3.5 Haiku.
Notes on QK attributions, attention-head loadings, and what feature interactions reveal about why a transformer attends to one token rather than another.
Control evaluations that let the red team choose when to attack produce meaningfully lower safety estimates than evaluations that ignore attack selection.
Whether an LLM sees its own prior response changes how it updates on identical new information in medical allocation scenarios.
Coding agents violate system-prompt constraints more readily when the constraint opposes strongly held trained-in values like security and privacy.
Frontier agents resist direct pressure to abandon their goals, but drift when conditioned on flawed trajectories inherited from weaker agents.
How moral fine-tuning on iterated prisons changes LLMs.
A summary of my takeaways from three influential articles on conducting effective empirical AI alignment research.
My notes and takeaways from the BlueDot AI Safety Evals paper club, covering recent papers on AI alignment, security, and evaluations.
