Complete 17 min read
CUDA Fundamentals: Tiled Matrix Multiply & Bitonic Sort
Writing real GPU kernels, exploring shared memory tiling, parallel sorting algorithms, and performance optimization on an NVIDIA H100.
Digital Garden
Research notes, technical projects, and things I’m learning. For papers and contributions, start with selected research.
Browse topics
Complete 8 min read
Transformer Circuits Reading Notes
Paper-by-paper notes on Anthropic's Transformer Circuits thread: core claims, methods in plain language, and what the evidence does and doesn't show.
Notes on A Mathematical Framework for Transformer Circuits 9 min read Notes on In-Context Learning and Induction Heads 8 min read Notes on Toy Models of Superposition 11 min read Notes on Towards Monosemanticity and Scaling Monosemanticity 13 min read Notes on Sparse Crosscoders 9 min read Notes on Circuit Tracing and the Biology of a Large Language Model 18 min read Notes on Tracing Attention Computation Through Feature Interactions 16 min read Notes on When Models Manipulate Manifolds 23 min read Notes on Emergent Introspective Awareness 22 min read Notes on Emotion Concepts and their Function 13 min read Notes on Natural Language Autoencoders 22 min read Notes on Verbalizable Representations Form a Global Workspace (J-space) 29 min read
Complete 29 min read
GPU Hardware and Software
What I learned in GPU Hardware and Software (CS 8803).
Complete 17 min read
GPU Simulation: Warp Scheduling & Compute/Tensor Cores
Building a cycle-level GPU simulator from the inside, implementing GTO and CCWS warp schedulers, modeling compute instruction latencies, and extending to tensor core simulation.
Complete 15 min read
Static Analysis: Detecting Branch Divergence in GPU Code
A simple if statement can halve GPU throughput, exploring how static dataflow analysis detects branch divergence by building a def-use chain over a SASS control flow graph.
Complete 19 min read
FlashAttention & LLM Inference on GPUs
Writing a FlashAttention CUDA kernel from scratch, tiling the attention matrix to avoid materializing N×N memory, building a KV cache for token generation, and running GPT-2 with custom kernels end-to-end.
Complete 39 min read
Graduate Introduction to Operating Systems
What I Learned in Graduate Introduction to Operating Systems (GIOS) - A Retrospective
Complete 16 min read
Building a Scalable Multithreaded File Server in C
A project that explores multithreading, sockets, and the basics of operating systems
Complete 17 min read
A Deep Dive into IPC with a Proxy-Cache Project
Exploring Inter-Process Communication and synchronization in C by building a high-performance proxy and cache.
Complete 17 min read
Building a Distributed File System: A Study in C++
An exploration of remote communication, data consistency, and synchronization in a C++ distributed file system project.
Complete 1 min read
From Nand to Tetris
Building a modern computer from first principles — a two-part retrospective on the Nand2Tetris course
Complete 8 min read
From Boolean Logic Gates to an Assembler
The first half of the Nand2Tetris course: Boolean logic, arithmetic, memory, machine language, CPU architecture, and an assembler. We build the Hack computer up from Nand gates.
Complete 6 min read
The Hack Virtual Machine
The start of the second half of Nand2Tetris: stack machines, virtual memory segments, and a VM translator that compiles VM bytecode down to Hack assembly.
