2026-04-12 | MOLT Followup on Auto Labelling
Goal: MOLT Gemma-3-1B-IT collapse fixed and Delphi AutoInterp Labelling + PCD Repro
Summary: Fixed training Collapse (MOLT PR #3), AutoInterp Failed because MOLT transforms to few (MOLT PR #4), finished pretraining code for PCD (Commit)
Work sessions
| In | Out |
|---|---|
| 00:00 | 01:30 |
| 10:30 | 14:30 |
| 18:00 | 22:30 |
Results for Eleuther/Delphi Labelling
- Results from using
Eleuther/Delphiare quite poor because the MOLT transforms used have a multiplier of N=1. Less transforms means despite being sparse, the MOLT transforms fire too often across too many unrelated contexts (and are thus not interpretable).