Skip to content

2026-09-28 | Maxwell

Goal: Add PP to Grammar + Map Landscape of Research Questions

Summary: My goal is to be more dilligent and clear with why I run experiments! Specifically focusing on what is the minimal experiment needed to answer a RQ I have and how it fits/bridges gaps existing experiments don't fit6

Work sessions

In Out Task
16:20 16:30 Planning
16:20 17:00 Grammar with PP only
17:00 18:30 Explain RQs and progress to ST
18:30 19:00 Finish Grammar with PPs and run training pipeline
19:00 21:30 Map of current research trajectories
22:00 22:20 Documentation

Goals

  1. If not DP Grammar leads to “hierarchical generalization” --> add Postpositionals into Grammar and re-run seeds

  2. Clarity

    • Write out on a piece of paper all the experiments that I’ve run so far

    • What is the key question I’m even trying to answer!?

    • RASP should probably fit in somewhere here

  3. Activation Patching experiments with different syntactic structures

  4. Pilot a Path Patching study

Meetings

  • Meet with Khalil: really great feedback about knowing why I'm doing experiments and not just doing experiments for the sake of having this vague notion of "progress"

Summary

  1. 🟢 [Running on cluster] If not DP Grammar leads to “hierarchical generalization” → add Postpositionals into Grammar and re-run seeds

    • Use PP attached only to matrix object and subject which use the ambiguous linear rule Clarity

  2. 🟢 Write out on a piece of paper all the experiments that I’ve run so far

  3. 🟢 What is the key question I’m even trying to answer!?

    • I think I’ve proved to myself that the 2 RQs I want to answer is:
1. Do LLMs use hierarchical representations in translation?

2. If so, how is the hierarchical representation constructed in the residual stream and used during translation?

3. What training pressures–specifically, training data complexity and diversity–lead to developing a hierarchical algorithm rather than a linear rule?

- RASP should probably fit in somewhere here

    - Perhaps RASP will fit in as bottom up approach to the top down casual abstraction; the more I think about this, the better I think this approach is
  1. [punt] Activation Patching experiments with different syntactic structures
  2. [punt] Pilot a Path Patching study