2024 letter

What does it mean to give a model a capability ? And while we’re on the subject, how do you give a model a capability? I’m talking about AI progress in the past year again. Contrary to the expectations that GPT-4 set in 2023, research labs emphasized the “capabilities” of the models they released this year over their scale. One example is “long context,” which is the ability to effectively process…