👁️ Effective Altruists Paid for Citations in Long-Term Intellectual Fraud Scheme

Welcome to Dark Markets, a weekly update on the Tech-right, Silicon Valley ideology, and related financial frauds from David Z. Morris, author of Stealing the Future: Sam Bankman-Fried, Elite Fraud, and the Cult of Techno-Utopia.

The tragedy of any book is that it is inevitably incomplete - and will stay that way. There are other things I wish I had been able to work into Stealing the Future, like new findings about Leverage Research and the story of longtermist fascist professor James Miller.

But nothing was a bigger miss than my omission of this scathing 2019 allegation of systematic intellectual fraud from Simon Knutsson, a PhD candidate in Sweden who is interested in suffering and existential risk, and worked closely with EA groups through the 2010s.

The short version of what he found - and the reason he departed the movement - were documents tying funding from EA organizations to specific intellectual conclusions, and to the citation of specific scholars. In other words, Knutsson essentially claims that organizations including Open Philanthropy were paying “scholars” to advance a specific agenda, with explicit guidelines for not merely topics, but intellectual frameworks, which would qualify for continued funding.

It’s impossible to overstate how scandalous this is. The EA “movement” has always presented itself as a series of think tanks that all happen to share the same concerns, agenda, ways of thinking. But it’s not a “movement” if you’re getting paid specifically to be a part of it, and it’s not “research” if your conclusions have already been dictated by your funders.

This entire intellectual trend, in so many words, is fake. It exists to serve the interests of the people funding it. That was fairly obvious from the outside, but Knutsson’s detailed report - including direct quotes from the intellectual loyalty oaths imposed on funded researchers - completely shatters the illusion that there was or has been any independent research at all conducted under the auspices of organizations like the Center for Effective Altruism or the Machine Intelligence Research Institute.

Crucially, Knutsson also observes that the people he worked with in EA “seemed nice and genuine. Most people I have worked with in EA seem nice.” When he communicates with Stefan Torges, one of the architects of the “communication guidelines,” Knutsson feels that Torges “is a nice guy, wants to help others, especially those who suffer, and he thinks that this is the best way to do so.”

But “this” is intellectual fraud.

EAs operated on the assumption that systemic corruption was the most “effective” way to promote their ideas, and so to save the world. In fact, they didn’t even see it as “corruption,” because the deeper utilitarian and eugenic authoritarianism baked into EA’s foundations eliminate the concept of a conflict of interest or broad intellectual discourse.

There was nothing wrong with paying people to promote their ideas because they were absolutely certain all of their ideas were right. Just like Sam Bankman-Fried would be.

The fraud detailed by Knutsson extends into traditional institutions. Both Will MacAskill and Toby Ord, who are professors at Oxford, helped craft the “communication agreements” that were signed among and between funders and organizations. Their own stature has been propped up, for nearly a decade now, by people being financially incetivized to approvingly cite their work.

This makes even more clear what I first wrote in 2023: these people should be removed from their academic posts. They are not scholars.

Knutsson gained access to documents detailing this scheme because he worked with these institutions for several years, volunteering for GiveWell as far back as 2010. He worked directly with Holden Karnofsky, who went on to head Open Philanthropy, the organization at the center of Knutsson’s allegations. Karnofsky departed Open Philanthropy in 2024. This was not publicly tied to his entanglement in the FTX scandal, but it’s hard not to connect the two. Karnofsky was one of three EA leaders who was warned about Sam Bankman-Fried’s unethical tendencies circa 2018, and then did nothing, directly enabling SBF’s larger fraud. Notably, Open Philanthropy has since rebranded to “Coefficient Giving,” possibly for similar reputational reasons.

Of course, all of these frauds ultimately enabled a lot of harm. Pretending EA was a legitimate research movement helped prop up Sam Bankman-Fried - who of course became the embodiment of EA’s fundamental ethical failure. But he wasn’t the source of that endemic corruption - only its expression.

As Knutsson puts it, “I think the public should be aware of these coordinated efforts behind the scenes that are partly aimed to influence which ideas about ethics and value are written about in public.” If only someone had listened sooner, a lot of destruction could have been avoided.

I will continue working to dig into the current status of Knutsson’s claims. It’s understandable that this went under the radar in 2019, before EAs and Rationalists had started doing embezzlement and spree murder at scale. But according to Andrew Orlowski at The Telegraph, similar practices have continued in “Progress Studies” and other fields that have followed EA’s decline.

There’s much more work to do here, but here are some of the specific claims made by Knutsson.

Two Documents: Guidelines for funded entities and individuals were laid out in formal documents shared and cross-endorsed between organizations. Many of these were seemingly written or guided by Nick Beckstead, an “AI Safety” researcher.

“The guidelines by Beckstead say, “EAF has written its own set of guidelines intended for people writing about longtermism from a suffering-focused perspective.” When a manager at EAF presented the two guidelines to me, he used phrases such as a “coordinated effort” and “it means that we will promote future pessimism … to a lesser extent.”

That hints at the fine intellectual conflict these documents intervene in. Very roughly, the rift was between pessimism and optimism about the long-term future. There’s a much deeper philosophical rift that won’t be unpacked here, but what this boils down to is EA organizations paying funded scholars to take sides in a debate on behalf of their own members.

They also specifically encourage citing specific people, and directly tied funding to that behavior. Beckstead most of all: “The EAF guidelines also encourage referencing Beckstead’s PhD dissertation …. I was told essentially that I could likely get funding in the future if I would follow the guidelines.”

This system ran through the entire EA think-tank complex. Elsewhere, Knuttson continues, “On July 18, 2019, Stefan Torges, one of the Co-Executive Directors of EAF, shared with me two communication guidelines in Google document format … [they were] ‘written by Nick Beckstead with input and feedback from various community members and several EA organizations. These guidelines are endorsed by the following organizations and individuals: 80,000 Hours, CEA, CFAR, MIRI, Open Phil, Nick Bostrom, Will MacAskill, Toby Ord, Carl Shulman…’”

That is to say, the most prominent figures in Effective Altruism looked at a pay-for-play scheme that tied citations and conclusions to grant money, and felt fine about it. The document’s appearance in ~2019 also coincided with a major grant between Open Philanthropy and the Effective Altruism Foundation (EAF). “That the $1,000,000 grant was made around the time the communication guidelines were shared makes the situation look even worse,” Knutsson writes.

But it’s not just amatter of suspicious timing - these were explicit, direct incentives for citation, and because EAs have such an underdeveloped sense of what’s actually right and wrong, they wrote it all down.

Furthermore, these “guidelines” were binding and extensive, covering even the social media posts of university employees who received EA funding.

This triggered Knutsson to immediately quit and sever ties. “On July 29 [2019], I proposed that I be removed from the FRI website, which I have been now, and I am no longer affiliated with EAF or FRI.” Knutsson worked at a real university, and probably risked some kind of ethics violation for taking what was essentially a bribe for citations.

Obviously this i wildly scandalous, to the extent that these organizations claim to be research organizations. What we’ve got here is a kind of ideological bload oath to present a unified front, to regurgitate a doctrine, to point to high priests. Knuttson reads it right: “I find all this to undermine research and public intellectual debate, and to be deceptive to those who are unaware of what goes on behind the scenes.”

This had potential direct consequences for Knutsson, who inferred that “the organisation where Jonas is Co-Executive Director receives 1 million in a grant for which Beckstead is a grant investigator, and then Jonas may discourage public discussion of my academic work that challenges moral views similar to Beckstead’s.”

I’ve reached out to Knutsson to catch up, but it seems pretty clear that objecting to this corruption has been bad for his career. He is still working on his PhD and talks about taking odd jobs around Sweden. Not something those still in the EA bubble have to struggle with!

He cites direct evidence of continuous pressure campaigns to defang scholarship critical of EA, SV longtermism, or its figureheads.

“Similarly, Torges wrote me on July 26, 2019, wondering if I had considered changing the title and/or abstract of my paper on the world destruction argument, which had recently been accepted for publication in a journal. He said he would be happy to help with this. I said no and that I react negatively to his question. … it was not just a friendly improvement suggestion. I mean, Stefan and others had just made an agreement with Beckstead, who was a grant investigator for a recent large grant to Torges’ organisation. I think Beckstead would like to see my paper not published at all or at least edited in certain ways. And then it seems like Stefan carried out that work of trying to influence this publication. That’s what I don’t like.”

Knutsson’s final judgment of the people he met through EA is understandably harsh. His 2019 assessment was that EA was a hive of deeply unethical and untrustworty people:

"If someone asked me which people in the whole world with a PhD in philosophy appear to behave in the most troublesome ways and which philosophers I trust the least, I would rank some of the philosophers in EA and existential risk circles around the top of my list."

History has now seemingly proven him correct.