Doing The Thing: From Mechanical Turk to Mech Interp

Updates

1. Currently investigating how the QK and OV projections in the Attention Mechanism represent and process syntax,the structure of sentences, through the task of translation from "SVO" Head Inital (e.g. English) to "SOV" Head Final (e.g. Japanese) languages [daily research journal, advised by Professor Khalil Iskarous and Professor Robin Jia]

2. Previously working on extending and reproducing Anthropic's MOLTs (sparse Mixtures of Linear Transforms) as part of Georg Lange's SPAR stream on Automating Circuit Interpretability with Agents (writeup, github)

3. Reproducing Transluce's Predictive Concept Decoders (PCD) to "read the mind" of an AI model through decoding activations of a Subject Model into natural language via a Sparse Encoder and LLM Decoder Model [github]

   • Started Karpathy-inspired video tutorials PCD reproduction line-by-line to make AutoInterp research more accessible [video walkthrough]

4. Playing with Fire: Proposal for a Sharing AI Safety Research with At-Risk Youth through Games [video, documentation]

   • 3-step pipeline to making AIS research accessible to broader community (e.g. at-risk teenagers using AI for mental health counseling); “AIS researchers are only one piece of the puzzle” ethos

5. Reflections from first 3 weeks of 2026 abroad at LISA and attending Oxford AI Safety Initiative's ARBOx including work in progress Theory of Change

TLDR of Things I've Done Since Starting My AI Safety Journey 362 Days Ago

Projects

1. Cross-Linguistic Alignment: Does LoRA Fine Tuning a model on a task (e.g. respond in all CAPS) translate cross-linguistically? (Summary && Github)

2. Reproducing Neo et al. 2024 Interpreting Context Look Ups

3. Multilingual Semantics Probe: Looking for Steering Vectors for semantically ambiguous sentences in English but not Mandarin

4. Syntactic Dependencies in Transformers: Attention Patterns for Balanced Parentheses (Dyck) Language (Github)

Programs

1. ARBOx: 2 weeks of compressed ARENA curriculum; project on Cross-linguistic generalization of fine-tuning

2. Attended NeurIPS Mech Interp 2025 Workshop and found some cool takeaways!

3. Started being mentored by Sudhanshu Kasewa from 80,000 Hours

What is this?

Here it goes! I'm Kyle, a 5th year undergrad/1st year master's in Computational Linguistics student at USC. I am very greatful to be jointly advised by Professor Khalil Iskarous and Professor Robin Jia.

Motivation

This is my daily log, doing-the-thing, a self-accountability + documentation + reflection in my journey in becoming a Mech Interp researcher.

If Language connnects us, how can studying language models at the intersection of linguistics and interpretability help us build human understanding of (a) how LLMs work and when they fail and (b) inform theories of real-time language processing in humans.

Burning Questions

1. What causal mechanisms allow language models to implement symbolic and discrete algorithms in distributed and continuous architectures? (e.g. syntax and translation)

2. How do these causal mechanisms emerge from training pressures, specifically the axes of training data and model architecture? (e.g. can we plot \(W_Q\), \(W_K\), \(W_V\) over time and intervene on gradients to control what circuits emerge?)

3. How can we create more faithful and accessible metaphors for general audiences to understand AI? (e.g. de-risk youth populations who confide in AI models for mental health counseling)

Extra Stuff (click me)

Wait What's Computational Linguistics?

As a Computational Linguistics student, I see Computational Linguistics as three parts:

1. Linguistics = study of human language processing / cognition
2. Mechanistic Interpretability = study of LLM language processing / cognition
3. Computational Linguistics = Interdisciplinary approach to studying LLM language processing

The Computational Linguistics topics that pull me at 9.8 m / s^2 are concepts like Information Theory and Probabilistic Phonology in addition to frameworks like Causal Abstraction and RASP (Restricted Access Sequence Programming).

What does Mechanistic Interpretability mean to me?

• I hope that having deep knowledge in both the fields of linguistics and ML/NLP can help me build a more holistic understanding of LLM cognition and language processing.
• I see Mechanistic Interpretability as a sort of psycholinguistics (the study of real-time processing of language) for LLMs.
• Furthermore, I see Mechanistic Interpretability as a foundational basis for understanding AI systems. Perhaps understanding models (such as like biological organisms) can support AI Safety, linguistics research, and provide more nuanced understanding of language models to the general public

Working Backwards

Sometimes (often) I get analysis paralysis or want to wait for the perfect {time, situation, background, preparation} to start which makes it difficult to get into pursuing my goals (and dreams). So this time around, I know my goal to become a Mech Interp researcher! After finding David Quarel's do the thing I decided that this site is a place where I will keep myself acountable for doing the thing.

• Doing --> Working through math problems, reading papers, writing down lists of possible intersections of linguistics and NLP
• The Thing --> Any of the above for at least 1 hour every day, with consistent (though not perfect) progress.

_{I had started doing the thing as a full-time student in my 4th year of undergrad. Mech Interp research is now my full-time priority (outside of TAing for a linguistics course) and far exceeds 1 hour a day}

What is dlog?

I am starting this daily log or dlog where every day I will document my progress. I hope that the daily act of documenting will make me more resilient and help prove to myself how badly I want to be an interpetability researcher. With the help of AI, a macro pulls the daily logs into the summary you see below:

dlog: 362 Days and Counting

Total time focused so far: 693 hrs 37 mins throughout 362 days of learning

Below are the latest updates (auto-generated).

Latest entries

• 2026-10-13 | 9 hr 30 min | Goal: Summarize new ideas from COLM and prepare to present at weekly meeting
  Tensor Product Representations, QK Decomposition, Variable Binding

  _{Maxwell · Syntax}

• 2026-10-12 | 2 hr 30 min | Goal: Write down ideas from COLM conversations
  Wrote down reflection with 9 new directions to pursue!

  _{Maxwell · Syntax}

• 2026-09-30 | 0 hr 15 min | Goal: Trying out different grammars and generalizations
  Using PPs does not help the model generalize right now, will revisit after COLM`

  _{Maxwell · Syntax}

• 2026-09-29 | 0 hr 0 min | Goal: Add PP to Grammar + Map Landscape of Research Questions
  My goal is to be more dilligent and clear with why I run experiments! Specifically focusing on what is the minimal experiment needed to answer a RQ I have and how it fits/bridges gaps existing experiments don't fit

  _{Maxwell · Syntax}

• 2026-09-28 | 5 hr 40 min | Goal: Add PP to Grammar + Map Landscape of Research Questions
  My goal is to be more dilligent and clear with why I run experiments! Specifically focusing on what is the minimal experiment needed to answer a RQ I have and how it fits/bridges gaps existing experiments don't fit

  _{Maxwell · Syntax}

• 2026-09-27 | 7 hr 45 min | Goal:
  __

  _{Maxwell · Syntax}

• 2026-09-14 | 0 hr 0 min | Goal: D-RASP Matrix Analysis | Associatve Memories with Outer Products
  Read Kevin Kirby's Rank One Memory Updates

  _{Maxwell · Syntax}

• 2026-09-12 | 0 hr 30 min | Goal: Read linguistic theories on syntax
  Thinking about workplace Syntax as a dynamical system

  _{Maxwell · Syntax}

• 2026-08-27 | 0 hr 30 min | Goal: Back from break! Onto planning and Causal Abstraction APIs
  __

  _{Maxwell · Syntax}

• 2026-07-28 | 1 hr 30 min | Goal: Raise PR for Intervention Interchange Accuracy (IIA) + Verify RASP-L Implementation
  __

  _{Maxwell · Syntax}