PhD Student at Cornell University — RL Environments, Post-Training & Agent Learning
Open weights are not just a model-release choice; they reshape who buys compute, who controls inference demand, and how much bargaining power enterprises have against closed APIs.
From LLM + tool use to context engineering, and then to long-running agent harnesses: agent capability is becoming a system property composed of the model, harness, context, tools, evals, sandbox, and state management.
In the age of agriculture, safety came from land; in the industrial age, from physical strength; before AI, from knowledge. Now knowledge suddenly feels cheap, and the old promise that knowledge changes destiny no longer feels like a stable truth.
After more than seven years of research, I am finally turning my attention back to learning. Without an upper bound or a verifiable reward, research made it impossible for me to know when to stop; letting it go has brought back a long-lost sense of peace.
Most abstention work treats 'should the model answer?' as a binary. We argue that's the wrong frame: an underspecified question wants clarification, a false-premise question wants correction, a time-sensitive one wants verification guidance — not the same generic 'I don't know'. We introduce the Helpful Abstention framework, a judge-based helpfulness × honesty (HH) evaluation across 15 open and 4…
We retrained Search-R1 across model sizes, RL algorithms, training distributions, search budgets, and broken-retriever settings — and ablated the scaffolding. The model's QA score barely moves when the think protocol is removed; it collapses when the retriever returns nothing; and the number of searches the model issues has almost nothing to do with the question. RL teaches the model to play the…
On the day of Meta layoffs. The office was nearly empty, people in WeChat groups were saying goodbye, and I started thinking about happiness deferred, the PhD that permanently changed my brain, and whether the bottom of the pyramid can disappear without the top collapsing too.
Continuing the conversation about reward from the previous post. On the gap between thinking and acting, the impossibility of forcing things, and learning to make decisions by vibe rather than by rule. If I get rejected, maybe fate is telling me this is just not the right place.
Late at night, half asleep, I suddenly realized how to solve an interview problem I had failed the day before. I had not had this kind of Eureka moment for a long time. Reflections on research, learning, exams, and why RL feels like an abstraction of life.
While sweeping reward compositions for our adaptive-reward paper, one configuration — Qwen3-4B trained with a HotpotQA-only judge — abruptly broke the SimpleQA leaderboard at training step ~400, jumping from 5% to 95% judged-correct in a few hundred steps. Across every other model × data × judge combination we tried, nothing like this happened again. Here is what we found.