Things that I want personally
(As opposed to wanting altruistically) General Biotechnology Societal interventions
musings and rough drafts
unpolished, semi-edited, streams of thought
Latest posts
Irreducible complexity does not preclude Napoleon, nor strategically superhuman AI
Dean Ball writes, in his post subtitled why I am not a doomer , that the blocker to AI takeover risk is computational irreducibility. A misaligned AI intent on taking over the world would fail because of the limits of intelligence: the intractability of predicting complex systems. I am doubtful about the ability of an AI Continue reading Irreducible complexity does not preclude Napoleon, nor…
Notes on the Caplan-Bruenig “poverty” debate
It seems like Bryan and Matt are mostly talking past each other. They re each advocating for changes along a different axis, and those two axes are in principle independent from each other. Bryan is primarily interested in the axis of the pervasiveness of market-restricting regulation in society (or alternatively how free are markets? ). He s advocating Continue reading Notes on the Caplan-Bruenig…
What’s up with the Anthropic board?
[Edit: I this post was based on a factual error. Reid Hoffman is not on the Anthropic board. Reed Hastings is. Thank you to Neel for correcting my mistake!] What are the dynamics of Anthropic board meetings are like, given that some of the board seem to not really understand or believe in Superintelligence? Reid Continue reading What s up with the Anthropic board?
Some questions that I have about AI and the overall strategic situation, and why I want to know
Will Automating AI R D not work for some reason, or will it not lead to vastly superhuman superintelligence within 2 years of ~100% automation for some reason? In what admin will the intelligence explosion occur? Will the arrival of powerful / transformative AI come from a lumpy innovation/insight? Will superhuman AI agents come out of Continue reading Some questions that I have about AI and the…
Frame: Rationality as self-hobbling
If most failures of rationality are adaptively self-serving motivated reasoning, choosing to be an aspiring rationalist is basically aspiring to a kind of self-hobbling. This is almost exactly counter to rationality is systematized winning. Suppose that we re living in a world where everyone is negotiating for their interests all the time, and almost everyone is Continue reading Frame: Rationality…
Why AIs aren’t power-seeking yet
[crossposted from LessWrong] Recently, I spent a couple of hours talking with a friend about the state of the evidence for AI takeover scenarios. Their trailhead question was (paraphrased): Current AIs are getting increasingly general, but they’re not self-promoting or ambitious. They answer questions, but they don’t seem to pursue convergent instrumental goals, for their Continue reading Why AIs…
Aspiring to rationality is choosing to be a purist
I m choosing to be an aspiring rationalist. That means that I m choosing to be a purist on a particular dimension. It s like honesty. Someone who is honest 99% of the time, but very occasionally (when it seems particularly high value or just when they feel like it) decides to lie to people instead is *not* Continue reading Aspiring to rationality is choosing to be a purist
Some further thoughts on Corrigibility and non-consequentialist motivations
[crossposted from LessWrong shortform] I think I no longer buy this comment of mine from almost 3 years ago. Or rather I think it s pointing at a real thing, but I think it s slipping in some connotations that I don t buy. What I expect to see is agents that have a portfolio of different drives and goals, Continue reading Some further thoughts on Corrigibility and non-consequentialist motivations
Humans are an evil god-species
Humanity, as a species, attained god-like power over the physical world and then used that power to create a massive sprawling hell. It obviously depends on where you draw the lines, but the majority of the participants of civilization, right now, are being tortured in factory farms. For every currently living human, there is currently Continue reading Humans are an evil god-species