2026-03-09 | IOI Transformer

Goal: Implement Transformer from Scratch | Train on IOI Task

Summary: Worked on guide for einsums and attentions pattern calculations

Work sessions

In	Out
08:50	09:50
10:30	11:50

See documentation on einsums and attentions pattern calculations

TODOs: 1. Use the same dataset strategy as Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers

1. Train SAEs

2. Train PLTs and CLTs?

3. Understand Attention Feature Interactions paper