2025-11-13 | Neo et al. 2024 cont.
Goal: Reproducing Neo et al. 2024 + Documenting + Apps
Summary: Applying to Research; implementing Section 4.1 Identifying Neurons (github reproduction)
Work sessions
| In | Out |
|---|---|
| 01:00 | 01:35 |
| 08:00 | 09:00 |
| 14:00 | 14:55 |
| 15:15 | 15:45 |
| 18:20 | 19:50 |
| 22:30 | 23:59 |
Neo et al. 2024
- Continued updating Interpreting Context Look Ups Notes
- Implementing Section 4.1
Identifying Neuronsbut found that the distribution of next-token neurons are highly skewed to the final layer in GPT2-Small and GPT2-Large. Additionally, applying RMS norm on \(W_{down}[:,i]\) did not reproduce the same Figure 3 as in the paper
Without Layer Norm:
With Layer Norm: (matches the Figure 3 better distribution but orders of magnitude different)
