ProgramBench Vetted: Reverse Engineering from a Runnable Binary

ProgramBench Vetted is a set of 50 tasks designed to test an agent's ability to rebuild a program from a runnable binary, with improvements to task design and validation. The benchmark aims to measure the agent's capability in long-context coordination and its ability to implement specified behavior without relying on shortcuts or exploiting defects in the evaluation environment. [summary]…