Alex Zhang's Website.
Harnesses can lead to compositional generalization: we observe a property in training RLMs, in which similarly structured tasks are viewed as isomorphic and all individual LM calls in the harness become in-distribution.
We study an example of the Mismanaged Geniuses Hypothesis at play on the LongCoT benchmark
We propose the mismanaged geniuses hypothesis, which posits that existing frontier language models are severely underutilized due to sub-optimal use of individual language model calls.
The language models we interact with in the near future will be what we call scaffolds today.
We propose Recursive Language Models (RLMs), an inference strategy where language models can decompose and recursively interact with input context of unbounded length through REPL environments.
A very long and thorough guide how deep learning algorithms, hardware, libraries, compilers, and more have become more efficient.
An annotated guide to the Kolmogorov-Arnold Network
Just me reading through every paper abstract...