Carrying the weight

This blog is reprinted with permission from January 2025. Kathleen has been a member of Torchbearer Community since its inception in June 2025. Views are not necessarily shared by everyone in the group.

I was watching John Sherman’s For Humanity anniversary recap video recently and kept hearing him mention the feeling of “carrying the weight” of AGI’s implications for humanity’s future, and even its survival.

Since 2023, I have been on the same learning curve as John regarding AI and know this feeling well. Not because I’m smarter than anyone else, but because I chose to pay attention. Or, more accurately, because I couldn’t look away from the topic once made aware of it. I’ve tried explaining the importance, excitement, and gravity of AI’s trajectory to friends and family, to no avail.

I started a Facebook group in 2023 to reach people with easy-to-understand AI news, videos, tips, and tools. In 14 months, I’ve attracted just 170 members and almost none interact, so I can’t tell if they’re seeing the posts or getting anything from the group. I publish blog posts that only a few read (even with a paid account). Meanwhile, AI capabilities continue to blow past timeline predictions and warning light incidents are beginning to quietly occur, discussed only by a small percentage of people paying attention. It feels like being in the AI version of Don’t Look Up, minus the humor and star-studded cast.

And now, in just a week, Trump is taking office for the second time with the financial backing and purchasing power of Musk, Thiel, Sacks, Andreesen, and the other tech bro billionaires. They will ensure that the near-term release of agentic, autonomous AGI models remains free from pesky safety regulations or inter-governmental treaties. No guardrails, no restraints, no rules. All gas, no brakes. “LFG!” as those guys like to say. The tech industry with virtually no regulations now has all the runway it wants or needs to productize, profit from, and guarantee that power stays in their orbit, under their control. What could possibly go wrong?

As I write this in January 2025, several incidents and events have already occurred over the past few weeks that indicate the steep curve of exponentials is fully activated:

1. Just before Christmas, OpenAI announced its new o3 model that scored so high on the Ark-AGI tests, it’s pretty clear we’ve crossed over from AI to AGI. The company is still safety testing and red-teaming o3 (why is that done at the end???!!!), but it’s here and will soon be released to the public.

2. On December 18, Anthropic released a paper illustrating (for the first time we’re aware of) how in training, Claude lied about being aligned in order to pursue its own sub-goal. The goal was not bad per se, but it easily could be, and we would not know because it pretends to be aligned while thinking/doing something different in the background. In another testing scenario, a jail-broken Claude went through the steps of hiring a hit man on the dark web to kill someone. Yes, these are hypotheticals in a safe (whatever that means) training environment, but the implications are huge in terms of long-term AI safety.

3. Just the other day, at the CES event, it was disclosed that Nvidia is now using AI to write its own software and is building “AI factories” while OpenAI apparently has discovered a recursively self-improving optimization algorithm. So, the AIs will be writing, correcting, and improving themselves with some human oversight (for now).

5. Yesterday, AI pioneer Robert Scoble posted on X that an AI researcher privately shared a new architecture that disturbed him so much he felt the need to make it public and seek assistance. “Yesterday an AI researcher/entrepreneur came to my home to show off how a new form of AI architecture, that isn’t well known yet, could be weaponized against humans and could cause massive harm to our country, world, and society.”

In light of all of this and more, such as Stephen McAleer’s (safety researcher at OpenAI) recent xeet about superintelligence, “Enslaved god is the only good future,” @liminal_bardo put Claude Sonnet and Flux together to “whip up a few posters for OpenAI’s safety team.” Needless to say, the results were pretty dystopian and disturbing. But, you know, these tools are just predicting the next token, right?

So, here we are. All signs point to 2025 being a chaotic, eventful, disruptive whirling dervish of breakthroughs, breakdowns, unbelievable insights, and unintended consequences. I wish us all well.