Paper Summary: A General Theoretical Paradigm to Understand Learning from Human Preferences

Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. A General Theoretical Paradigm to Understand Learning from Human Preferences Source WHAT Authors propose a generalized framework called Ψ-PO to represent generic differentiable functions that can directly optimize preference data (similarly to DPO). DPO and RLHF are special…