Sigmoids all the way down
A few thoughts on modeling exponential and saturating growth
Facilis descensus ad minimum
A place to discuss the gory details of the current state of machine learning research: what works, what doesn’t, and why we don’t have AGI yet.
Live Last read · last published · next check
Latest posts
Moravec's paradox applies to genetics, too
And other implications of adaptivity
The state of plasticity in 2025
A survey
A visual guide to proximal methods
Extra gradients, hold the momentum
Does parameter norm growth cause networks to lose plasticity?
A brief clarification
Paper review: Auditing language models for hidden objectives
What's in an objective?
Can you train a neural network forever?
Understanding trainability in neural networks
Neural networks live life on the edge
(of stability)
Do we know why deep learning generalizes yet?
It depends on your epistemology
When do adaptive optimizers fail to generalize?
A case study