Skip to content

2026-06-03 | Maxwell

Goal: Interpret Attention Heatmaps of Toy English <-> Zapanese Models

Summary: Working on understanding Cross Attention for heatmap evaluation

Work sessions

In Out Task
10:30 11:20 Vaswani et al. Hand Implementation
16:30 17:20 Vaswani et al. Hand Implementation

Goals

As a reflection, putting this many goals is one day is kind of unrealistic and quite hard to figure out!

  1. [P0] Multiple training initializations

  2. [P0] Test for Grokking

  3. [P0] Add Adjectives, Adverbs, and Relative clauses to Grammar

  4. [P1] Shrink model size (e.g. 2 layer Attention only model)

  5. [P1] Generalized tool to check against specific attention pattern

  6. [P2] Activation Patching

  7. [P2] Literature Review

    a. Polar Probe

    b. “How do Language Models Bind Entities in Context?”

Meetings

  • Send out meeting reminder email to professors

Summary

  1. ✅ [P0] Multiple training initializations

  2. ✅ [P0] Test for Grokking

  3. 🟡[P0] Add Adjectives, Adverbs, and Relative clauses to Grammar

  4. ✅ [P1] Shrink model size (e.g. 2 layer Attention only model)

  5. 🟥 [punt for now] [P1] Generalized tool to check against specific attention pattern

  6. 🟥 [punt for now] [P2] Activation Patching

  7. 🟥 [punt for now] [P2] Literature Review

  8. “How do Language Models Bind Entities in Context?”