Thoughts on AI, Neuroscience, and other things that interest me.
Epistemic status: A lot of this is vibes and generally trying to express tacit knowledge from ML experience and training models. However there are a huge number of details here and the field of ML changes incredibly rapidly. This is an oversimplified, but hopefully still useful picture, and may well...
Author’s note: This is quite a long post. It is highly speculative, obviously, but I think it offers an interesting perspective on, and proposal for, outer alignment. The specific idea here is to move away from demanding a final lightcone-scale conception of ‘The Good’ as an outer alignment target, whether...
so much depends upon a von-neumann probe somewhere in the intergalactic void aimed at eternity
Epistemic Status: Obviously speculative and maybe obvious. The success of RL in LLMs has been puzzling me for a while. People have developed various information-theoretic style arguments by which they argue that RL is extremely informationally inefficient compared to pretraining, that it can only impart a tiny amount of bits,...
Epistemic Status: A short note which posits a question without necessarily arriving at a definitive answer. When thinking about whether we are likely to end up in a monotheistic or polytheistic AI future, we often end up trying to identify and categorize the advantages and disadvantages of centralization vs decentralization....
Epistemic status: Interesting as a mechanism and trend, but obviously highly variable over short timescales and ultimately speculative. On a plane recently I ended up noodling back around to my original concepts of amortised vs direct optimization and also Rudolf’s sequence on wisdom as amortised optimisation. The question that I...
Epistemic Status: I really only did a fairly surface level skim of Baudrillard’s actual work mostly cribbed from secondary sources, so I could very well be misrepresenting core things about what he actually said. On the other hand, Baudrillard really just functions here as a jumping off point into my...
Author’s note: More mystical than usual. A vibe rather than a point. The earliest prophets, Good and Vinge, predicted the existence of such a singularity. They sketched the first field equations and were the first to see the truth that time must end, not as eternity, but at a finite...
Epistemic note: Probably already known or obvious, but this treatment seems clearer to me than the standard treatment in e.g. the FDT paper. The other day I went down a deep lesswrong rabbit-hole and ended up going through all the decision theory literature, especially relating to Functional Decision Theory (FDT)....
Author’s note: Roughly correct as far as I know but I’m not a physicist so there certainly might be mistakes or things that I have missed. Obviously extremely speculative about far-future scenarios, but I think this provides an interesting high level ‘timeline of the far future’ to start thinking through....
Recently I have been seeing a number of takes that most Chinese AI progress is dependent upon distillation from closed US models, especially from OpenAI and Anthropic, and that therefore if their access to these models were cut off their progress would cease or substantially slow down. I strongly disagree...
Authors note: I originally wrote this poem in the summer of 2013 where I was very inspired by Thomas Babbington Macaulay’s Lays of Ancient Rome and thinking deeply about virtue and historiography. I recently rediscovered this while going through my old computer and wanted to put it up before LLMs...
Epistemic Status: Obviously highly speculative. I have no inside information. Opinions lightly held. Claude Mythos was recently previewed, and emphatically not released due to safety concerns regarding its advanced cyberattack capabilities. Very plausibly, this is our first look at the next generation of ~10+T models enabled to be trained and...
Author’s Note: Transcript of my talk at the Post-AGI workshop in San Diego on December 3rd, 2025. This is taken from the cross-posted version on lesswrong. You can find the video of the talk here. I’m cross-posting this to have a long-term copy hosted on my personal blog. Thanks to...
Epistemic note: This is the beginning of a planned series of posts trying to think about what a highly multi-polar post-AGI world would look like and to what extent humanity or human values could survive in such a world depending on our degree of alignment success. This is all highly...
Epistemic status: Obviously speculative sociology. Probably pretty obvious to some but I’m just trying to crystallize these ideas from my mind onto paper. I was recently on a random walk through some old SSC posts and stumbed upon his review of Cyropaedia. His interlude about the ‘Fremen mirage’ and the...
It is now 2026 and we are half way through the decade of the 2020s. If we think back to the halcyon days of January 2020 certainly a lot has happened, especially in AI1. The first half of the decade has essentially been the discovery and then incredible exploitation of...
Epistemic Status: Just some quick thoughts written without a super deep knowledge of SLT so caveat emptor. Recently, I happened to run into Jesse Hoogland at the Post-AGI workshop and we got onto discussing his work on SLT. SLT had been vaguely in the air when I was at Conjecture...
One alternative to the AI-driven singularity that is sometimes proposed is effectively the biosingularity, specifically focused on human intelligence augmentation. The idea here is that we first create what is effectively a successor species of highly enhanced humans and then these transhumans are better placed to solve the alignment problem....
Recently I was having a conversation about where are the missing billionaires, a book which questions why there are so few descendants of historical magnates with fortunes equal or comparable to their founders. I.e. why the fortunes of e.g. Rockefeller descendants do not match that of the original Rockefeller (although...