Virtually nobody is pricing in what's coming in AI. I wrote an essay series on the AGI strategic picture: from the trendiness in deep learning and counting the OOMs, to the international situation and The Project.
A new research direction for superalignment: can we leverage the generalization properties of deep learning to control strong models with weak supervisors?
We’re launching $10M in grants to support technical research towards the alignment and safety of superhuman AI systems, including weak-to-strong generalization, interpretability, scalable oversight, and more.
AGI will effectively be the most powerful weapon man has ever created. Neither “lockdown forever” nor “let ‘er rip” is a productive response; we can chart a smarter path.
Society really cares about safety. Practically speaking, the binding constraint on deploying your AGI could well be your ability to align your AGI. Solving (scalable) alignment might be worth lots of $$$ and key to beating China.
Far fewer people are working on it than you might think, and even the alignment research that is happening is very much not on track. (But it’s a solvable problem, if we get our act together.)
America’s economic dependence on China creates a security vulnerability. But tariffs are royally ineffective at mitigating this vulnerability. I consider the underlying informational problem.