Omar Khattab
14.1K
posts
@lateinteraction
Cambridge, MA
Joined December 2022
oh that’s a GREAT take
Replying to @lateinteraction
Interesting view. Brain does agentic attention and sys1 recurrence. Transformer-based agents do the opposite.
Intuition: Compaction is agentic recurrence (RNNs), whereas recursion (RLMs) is agentic attention. Recurrence maintains a constant-size state at all times by compressing the past. Attention keeps all context fully represented and then decides how to re-process it for each step.
part of this is in how ~everyone with a platform here speaks about LLMs for the past few months i rely on LLMs as excitedly as anyone, but... are we using the same models, guys?! the frontier is still dumb as a brick 20%+ of the time and needs more hand-holding than a freshman
The vibe and language here have been drifting wholesale away from sensible, nuanced takes on the technical topics that animate me. Been oscillating on whether I should essentially just check out other than amplifying releases, as I've done for the bulk of this year to date.
LLM review weirdness... Just renaming an uploaded pdf from "paper.pdf" to "paper_final_draft_pdf_ready_for_review.pdf" boosts average scores (gpt-5.6-terra)
The vibe and language here have been drifting wholesale away from sensible, nuanced takes on the technical topics that animate me. Been oscillating on whether I should essentially just check out other than amplifying releases, as I've done for the bulk of this year to date.
