Skip to content

2026-09-30 | Maxwell

Goal: Trying out different grammars and generalizations

Summary: Using PPs does not help the model generalize right now, will revisit after COLM`

Work sessions

In Out Task
08:45 09:15 Really fun conversation with Prof. TM on Subject vs. Object Asymmetries
09:15 10:30 Document experiments + contextualize in RQs
10:30 11:00 Meet w/ RJ to get feedback on presentation
13:00 11:00 Document experiments + contextualize in RQs

Goals

  1. If not DP Grammar leads to “hierarchical generalization” --> add Postpositionals into Grammar and re-run seeds

  2. Clarity

    • Write out on a piece of paper all the experiments that I’ve run so far

    • What is the key question I’m even trying to answer!?

    • RASP should probably fit in somewhere here

  3. Activation Patching experiments with different syntactic structures

  4. Pilot a Path Patching study

Meetings

  • Meet with Khalil: really great feedback about knowing why I'm doing experiments and not just doing experiments for the sake of having this vague notion of "progress"

Summary

  1. 🟢 [Running on cluster] If not DP Grammar leads to “hierarchical generalization” → add Postpositionals into Grammar and re-run seeds

    • Use PP attached only to matrix object and subject which use the ambiguous linear rule Clarity

  2. 🟢 Write out on a piece of paper all the experiments that I’ve run so far

  3. 🟢 What is the key question I’m even trying to answer!?

    • I think I’ve proved to myself that the 2 RQs I want to answer is:
1. Do LLMs use hierarchical representations in translation?

2. If so, how is the hierarchical representation constructed in the residual stream and used during translation?

3. What training pressures–specifically, training data complexity and diversity–lead to developing a hierarchical algorithm rather than a linear rule?

- RASP should probably fit in somewhere here

    - Perhaps RASP will fit in as bottom up approach to the top down casual abstraction; the more I think about this, the better I think this approach is
  1. [punt] Activation Patching experiments with different syntactic structures
  2. [punt] Pilot a Path Patching study