Paper Summary: KTO: Model Alignment as Prospect Theoretic Optimization

Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. KTO: Model Alignment as Prospect Theoretic Optimization Source WHAT Authors develop a framework with HALOs (Human-Aware Losses), which add insights from Prospect Theory 1 to losses used to instruction-tune LLMs in DPO and similar strategies. They introduce one specific HALO…