Skip to content

2026-05-28 | Maxwell

Goal: Add Bare Nouns (e.g. 'John likes Betty') to not overgenerate ('The John likes the Betty')

Summary: Allow CFG to support recursively embedded relative clauses (e.g. 'John knows that Betty likes that Tom sings')

Work sessions

In Out Task
09:00 17:00 Design Toy Grammar to support Bare Nouns

Goals

  1. [P0] Design grammar to contain relative clauses, adjectives, and adverbs

  2. [P0] Train existing Vaswani et al. Transformer on (a) Depth {0,1,2} with holdout depth {3} and (b) uniformly sample depths {0,1,2,3}

  3. [P1] Train Toy GPT-2 Small style architecture on (a) and (b) above

Others

  1. Complete required REU Survey

Meetings

  1. None

Summary

  1. ✅ Designed grammar to include Bare NPs (e.g. “John”), Transitive, Intransitive, and CP specific verbs

  2. 🟡 [in progress] [P0] Train existing Vaswani et al. Transformer on (a) Depth {0,1,2} with holdout depth {3} and (b) uniformly sample depths {0,1,2,3}

  3. 🟡 [not started] [P0] Train Toy GPT-2 Small style architecture on (a) and (b) above

Additional

  1. Added TODO: understand how RoPE works

  2. Added zsh and oh-my-zsh plugin to user in cluster