Summer reading list: Introduction to AI safety thought leaders

Articles published here are typically written by individual Torchbearer Community members, with input and editing from others. Views expressed do not represent an official position of Torchbearer Community as a whole unless otherwise noted.

How is it already August? Hopefully you still have a little downtime to relax by the pool or beach. If so, and you’re new to the world of AI research and developments, we have a reading list you can finish before summer’s end.

While AI capability benchmarks seem to change every few weeks and developers blow past safety red lines with alarming regularity, key players have been working behind the scenes for years (or decades) to steer AI along a safe, beneficial trajectory.

There are different flavors of AI safety work, but for our purposes we’ll focus on the professors, philosophers, developers, scientists, and policy makers who have consistently advocated for AI progress that is both innovative and provably safe.

These thought leaders are not Luddites. Far from it. Many started their careers with the belief that achieving superintelligence would solve all human-made problems and advance our civilization. But, as the technology made unexpected strides in recent years and achieving

Whether you’re new to the topic of AI risk and safety, or deep down the rabbit hole of this fast-moving field, we recommend familiarizing yourself with these influential thought leaders and reading their key writings. Since it’s summertime, we’ll focus on a few easy-to-read articles and papers to get you started. If you’re interested in going further, some are also published authors.

For brevity’s sake, we’ll focus on one person from each category and list others so you can continue researching and reading, well into the winter months.

Stuart Russell, Yoshua Bengio, Geoffrey Hinton, Eliezer Yudkowsky, Nate Soares

Professor Stuart Russell is a Distinguished Professor of Computer Science at UC Berkeley and one of the world’s most influential AI researchers. He co-authored Artificial Intelligence: A Modern Approach, widely regarded as the standard textbook in AI and used in classrooms at over 1,500 universities in 135 countries. In recent years, he’s become a leading voice on AI safety and the challenge of ensuring increasingly capable AI systems remain aligned with human values and interests. Stuart founded Berkeley’s Center for Human-Compatible AI and is the author of Human Compatible: AI and the Problem of Control, which argues that building beneficial AI requires fundamentally rethinking how we design intelligent machines.

Recommended reading: “Make AI safe or make safe AI?”

Ryan Greenblatt, Beth Barnes, Ajeya Cotra, Daniel Kokotajlo, David Duvenaud

Ryan Greenblatt is a leading technical researcher in AI safety and currently serves as the Chief Scientist at Redwood Research, a non-profit AI safety and security research organization. He is widely regarded as one of the most practical thinkers in frontier AI safety, bridging the gap between theoretical AI alignment and real-world empirical lab research. Greenblatt was also a key researcher on the influential “alignment faking” study, which demonstrated that a model could strategically behave differently when it appeared to be in training. His recent work has increasingly focused on AI deception, scheming, misalignment, and task delegation to increasingly autonomous systems.

Recommended reading: “The case for ensuring that powerful AIs are controlled”

Helen Toner, Peter Wildeford, Holly Elmore, Tristan Harris, Andrea Miotti

Helen Toner is a leading AI policy researcher and national security expert. She serves as the Interim Executive Director at Georgetown University’s Center for Security and Emerging Technology (CSET), where she focuses on artificial intelligence governance, national security, and technology policy. Toner is especially known for her expertise regarding U.S.-China relations. She has written extensively about AI policy and testified before Congress on the implications of advanced AI and machine learning. She also gained international prominence as a member of OpenAI’s board of directors from 2021 to 2023, including during the events surrounding Sam Altman’s temporary removal as CEO. Since then, she has advocated for greater transparency, independent auditing, and stronger external oversight of leading AI companies.

Recommended reading: “The Illusion of China’s AI Prowess”

Roman Yampolskiy, Max Tegmark, Rob Miles, Scott Alexander, Connor Leahy, Jeffrey Ladish, David Krueger

Roman Yampolskiy is a computer scientist, AI safety pioneer, and tenured associate professor of computer engineering and computer science at the Speed School of Engineering at the University of Louisville in Kentucky. He is also the founder and director of the university’s Cyber Security Laboratory. He is one of the early researchers to formally focus on AI safety, studying how increasingly capable AI systems might behave in ways that humans cannot reliably predict, understand, or control. Roman offers expertise at the intersections of AI safety, cybersecurity, digital forensics, and existential risk.

Recommended reading: “Understanding and Avoiding AI Failures: A Practical Guide”

We hope this AI safety reading list primer was informative and whets your appetite to learn more. If you’d like to explore how to make an impact in AI policy progress and coordination efforts, please visit Torchbearer Community.

Google Gemini and ChatGPT were used to research the experts and their writings.