Recent content on Haozhe Jiang
DiPOD: Diffusion Policy Optimization without Drifting Apart Diffusion language models are an exciting alternative to autoregressive language models. Instead of generating a response strictly left-to-right, they can refine many tokens in parallel. This opens the door to faster sampling, flexible decoding orders, and possibly new ways of doing long-form reasoning. But there is a gap between this…
Large Language Models are Surjective? Injective? Invertible? Recently, there have been discussions on functional properties of Transformers, the basic building block of Large Language Models (LLM) and many other generative models. My paper ([1]) proves that Transformers can output anything given an appropriate input (surjective). After a few months, a followup ([2]) proves that LLMs always send…
Diffusion Policy Optimization without Drifting Apart Haozhe Jiang, Haiwen Feng, Jiantao Jiao, Pieter Abbeel, Angjoo Kanazawa , Nika Haghtalab Reinforcement Learning Diffusion Model Generative Model We develop a framework to analyze policy gradient for diffusion models, while preserving the structure of diffusion models. We identify the 'double drift' phenomenon when one replaces the log-likelihood…