Skip to content

2026-07-22 | Maxwell

Goal: Verify RASP-L Implementation Gemma3-4b-it algorithm

Summary: Try using instruction-tuned Gemma model to run translation experiments

Work sessions

In Out Task
09:30 12:00 Gemma
13:30 15:00 Outer Products Lecture
15:00 16:00 Hypothesis Generation Pipeline Lecture
16:00 17:00 Gemma

Goals

  1. Gemma3-4b-it Causal Abstraction with different English words
    • Refactor into experiment 2
  2. Understand how a DAS experiment would work (what to hypothesize as a variable in H → N)
  3. Depth (keep the same recursive structure but then just swap the “that” activation somewhere else)

Meetings

  • Iskarous 1:30pm
  • Robin 3pm

Summary

  1. 🟡 [in progress] Gemma3-4b-it Causal Abstraction with different English words
  2. 🟢 Understand how a DAS experiment would work (what to hypothesize as a variable in H → N)
    • Realized a mistake in the RASP-L program; uses bidirectional encoder instead of a causal decoder style

Consensus from professors that toy models will be super cool way to validate pipeline and icing on top is pretrained/instruction tuned models