2025-11-28 | Reproducing Neo et al.
Goal: Reproduce Neo et al. 2024
Summary: Add support for extracting prompt activations, in progress streaming in The Pile dataset
Work sessions
| In | Out |
|---|---|
| 15:00 | 16:30 |
| 20:15 | 21:30 |
Back from a nice Thanksgiving Holiday! This is the first time of having a streak break but ready to continue working!
Reproducing Neo et al.
- In progress code
- Created a phi function to calculate max neuron activation for a prompt/layer
- Using a top-k heap for sorting the highest (k=20) activating prompts for neuron
- Ramping up onto streaming
The Piledataset - using the uncopyrighted version and using streaming for space efficiency