Evaluating Long Context (Reasoning) Ability

Pass@1 scores on the 128k subset of LongCodeEdit. Reasoning models and long agent trajectories are eating up valuable space in the context window. In response, models are being released with ever-increasing context windows; the latest, Grok 4 Fast, has a 2 million token window. Unfortunately, as anyone who has worked with these models knows, the number of tokens a model can accept as input is not…