arena-in-12-weeks
glossary about

a 12-week guided curriculum

ARENA in 12 weeks

From "I can write Python but I've never trained a neural network" to "I can find and causally verify circuits inside GPT-2" — the official ARENA 3.0 exercises, repackaged for a part-time study group with no ML background assumed.

how it works

During the week you read that week's explainer here — plain-English concept building, a glossary, and small runnable code blocks. No TAs: you're paired into a standing group of 2–3, and once a week your group meets for 3–4 hours to work through the official ARENA notebook together in Colab.

It's fine not to finish a notebook. Each week marks which sections are core and which are stretch; when stuck, the solutions notebook is always legitimate — read it, then re-derive the answer.

the 12 weeks

  1. 01

    Tensors, einops & einsum

    Everything in deep learning is an array with a shape.

    arena 0.0 · act i · foundations
  2. 02

    Your first neural network

    A neural network is ordinary code you can write yourself.

    arena 0.2 · act i · foundations
  3. 03

    How training actually works

    Training = rolling downhill on a loss landscape.

    arena 0.3 · act i · foundations
  4. 04

    Backprop from scratch

    Gradients come from the chain rule on a computational graph — no magic.

    arena 0.4 · act i · foundations
  5. 05

    Build a transformer

    Attention: tokens looking at other tokens.

    arena 1.1 · act ii · transformers
  6. 06

    Opening the box: induction heads

    Real transformers do their work via attention-head circuits.

    arena 1.2 · act ii · transformers
  7. 07

    Superposition & toy models

    Features ≠ neurons — models pack more concepts than they have dimensions.

    arena 1.5.4 · act ii · transformers
  8. 08

    SAEs on real models

    Dictionary learning recovers human-readable features from real LLMs.

    arena 1.3.3 · act ii · transformers
  9. 09

    The IOI circuit & activation patching

    Causal evidence: prove a circuit does the job by editing activations.

    arena 1.4.1 · act iii · toolkit
  10. 10

    Linear probes

    You can read a model's “beliefs” straight out of its activations.

    arena 1.3.1 · act iii · toolkit
  11. 11

    Steering & function vectors

    Once you've found a direction, you can push the model along it.

    arena 1.3.2 · act iii · toolkit
  12. 12

    Capstone: grokking & where to go next

    Models learn real algorithms — and you can fully reverse-engineer one.

    arena 1.5.2 · act iii · toolkit

prerequisites

Comfortable Python (functions, classes, comprehensions) and high-school maths. Anything beyond that — matrix multiplication, derivatives, log/exp — is taught when needed. You'll need a Google account for Colab; nothing is installed locally.