AI safety is often framed as a technical puzzle. How do we align models with human values? How do we detect deception? How do we prevent loss of control?
But beneath these technical questions lies a deeper structural reality: building frontier AI now costs billions. And whoever controls those billions shapes what gets built, how fast, and with what safeguards.
The capital-safety nexus
As AI development becomes more capital-intensive, labs take on larger compute commitments, higher valuations, and stronger obligations to investors, employees, and customers. These pressures do not need to produce explicit demands to ignore safety. They operate more subtly: they shape which research gets funded, which risks are taken seriously, how quickly models are released, and whether a company can credibly pause when doing so would cost billions.
A lab that has raised $10 billion cannot simply “pause” without consequences. Its investors expect returns. Its employees expect compensation. Its customers expect delivery. These are not bad actors making bad choices—they are rational actors responding to the incentives that capital creates.
This raises fundamental questions: who controls the resources required to build frontier systems? What returns do those investors expect? And who has the authority to sacrifice economic value in the name of safety?
Voluntary commitments are weakest when they matter most
The industry has produced numerous safety commitments: voluntary pledges, framework agreements, and public promises to pause if certain thresholds are met. But there is a structural pattern: these commitments are weakest precisely when they would be most costly to honour.
Several developers have now made pause commitments entirely contingent on competitors’ behaviour—a provision that ensures no one ever pauses alone. The Future of Life Institute’s Summer 2026 AI Safety Index found safety frameworks that rarely specify quantitative thresholds, independent audit, or who can halt a deployment.
Good intentions are not enough. They cannot override the structural pressures of capital markets.
What effective safety requires
If safety is genuinely to matter, it cannot rely on voluntary goodwill. It requires control over capital, incentives, and decision rights. This means:
Investors who ask hard questions about safety before writing cheques
Governance structures that empower boards to halt deployments
Regulatory regimes with binding enforcement powers
Capital allocation that rewards safety, not just speed
These are not technical solutions. They are institutional ones. And they are the ones that will ultimately determine whether frontier AI develops safely or recklessly.
Join us for this conversation
Jenny Xiao, co-founder and General Partner at Leonis Capital, joins AI Safety Hong Kong to reframe the safety debate through the lens of capital markets and corporate governance. Drawing on her unique perspective—from early research at OpenAI to leading a research-focused VC fund—she will explore why voluntary safety commitments fail, how financing structures shape organisational behaviour, and what it would actually take to build safety into the system.
This talk is designed for investors, founders, board members, policy professionals, and anyone who wants to understand the financial realities driving AI development. No technical background required.
About the Speaker:
Jenny Xiao is the co-founder and General Partner at Leonis Capital, a research-focused VC firm based in Silicon Valley. Prior to Leonis, she was an early employee and researcher at OpenAI, and holds a Ph.D. from Columbia University. Her research has been featured in Nature, The New York Times, The Wall Street Journal, The Economist, Fortune, and CNBC. Her work at Leonis is the subject of an extensive Harvard Business School case study, and she is a frequent speaker at leading industry conferences including Fortune and SuperReturn. She is also a recipient of the Forbes 30 Under 30 award.
Event Details
📅 Date: 18 August, 2026 (12:00pm HKT)
📍 Location: Online
🔗 Register: https://luma.com/9seock3e
Stay connected with AI Safety Hong Kong
LinkedIn: https://www.linkedin.com/company/ai-safety-hong-kong/
Substack: http://substack.com/@aisafetyhk
Website: https://www.aisafetyhk.org/
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.