On numbers, friendship, and violence

For those of us who believe A Really Bad Thing is happening, a common counterargument is “Oh, but if you really believed that, you wouldn’t be acting in anything like the normal way you do; probably acts of extreme violence would be justified. Reductio. Therefore A Really Bad Thing is not happening,” though usually with that last part left implicit. You see this, especially, with the two most important things ever, namely the unconscionable crimes inflicted upon animals just to be eaten, and the very threat of human extinction from superintelligent AI. And so we obtain an infallible proof completely independent of experience of the impossibility of most people committing acts of torture and murder, and the impossibility of humanity going extinct from new technology—a class of contingent a priori truths hitherto unknown to philosophy.

bone@boneGPT

remember: if they actually believe all this stuff and they are unwilling to be violent, it means they are cowards, that they refuse to measure up to their own words, that they will not do what they believe needs to be done to save mankind. they are weak, they believe in nothing

Trevor Bingham @22TrevorBingham

Our response to the Molotov attack on Sam Altman's house is unequivocal condemnation with no excuses or apologies for the deranged person who did this. The perpetrator committed an indefensible crime and clearly suffers from profound mental illness; no rational person firebombs

3:13 PM · Apr 11, 2026 · 7.04K Views

7 Replies · 1 Repost · 61 Likes

Two things should be clear: first, this is not how you reason when you want to be correct. You cannot figure out if empirical claims—such as those about the risks of AI—are true by seeing what actions the claims would justify if true, as if God ordained from the beginning of the universe that circumstances would never get too nasty. Nor can you use the same reasoning to deduce that a large number of people aren’t culpable for murder and torture. It is not hard to imagine a world where that is true, and if you want to claim that that is not our world, then you have to figure out if people’s actions are permissible first, unless you think God preordained that large numbers of people won’t do sufficiently evil things.

Second: basically no one engages sincerely in the sort of reasoning just described. Perhaps I’m being optimistic, but it’s too obviously a form of wishful thinking to be believable. Rather, people who make these sorts of arguments just want to shut you up and give themselves an excuse to substitute bad heuristics for actually thinking about the topic at hand. It’s manipulative, it poisons the mind of everyone who allows themselves to engage in such ways of thinking, and it’s the kind of thing we should be constantly on guard against.

I do not doubt, however, that plenty of people really do think that believing A Bad Thing is happening would justify violence, and that this partly explains some people’s fear of engaging with these matters rationally. In some superficially similar cases, violence really is justified; if you were a citizen of Nazi Germany, then yeah, there would be no moral problem with committing acts of violence against the government (though it might not be expedient). Seeing why the cases are disanalogous would probably require thinking about what it’s like to be an ethical vegan or someone worried about AI risk for a few minutes at least, and people by and large do not spend multiple minutes trying to reason conditional on views they disagree with.

Rather, they think, “If these people are right, then eating animals is morally equivalent to what the Nazis did; if that’s true, then it would be justified to treat people the way Nazis should be treated,” and then consider the matter settled without much further thought. Less extremely, people also argue, “You don’t treat people who eat animals the way you’d treat a Nazi; at the very least, if you really believed what you say, you wouldn’t be friends with them, you’d want nothing to do with them, you’d even disown your family.” I would wager that people who say this have not tried to imagine a case that’s actually analogous; where, say, 98% of people are unquestioning Nazis, not out of passionate hatred but as a result of not having seriously considered anything else, where the 2% are just not taken seriously, where you yourself are a former Nazi, etc. Would you commit individual acts of violence in such a scenario, or even treat basically everyone the way you’d treat a neo-Nazi? Should you, morally speaking? The correct answer to the former question is “no,” and the same is true in the latter case, or at least I claim.1

To be clear, I claim using violence to oppose the animal holocaust or AI risk is morally wrong, and not merely inadvisable because it would be ineffective (though the latter is also true). People are right to be mindful of that distinction. To back the moral claim, however, I’m afraid I’ll have to presuppose certain ethical truths, as I don’t think the case can be made more ecumenically. If utilitarianism is true, for instance, then such violent acts probably are merely imprudent, and not in principle wrong.

Ethics, as I have said before, is about when (if ever) you should do something other than what you already want to do. If it’s always permissible to act as your antecedent preferences would dictate, then we don’t need ethics, we only need to figure out what’s expedient. But as far as we have substantive duties, and not merely desires that we call by that name, then we assume there might be a case where what you ought to do is really, truly something you do not want to do.2

People like the idea that ethics is independent of desire when stated in the abstract, I think because the case they imagine is that of a sociopath, and they want to say that even if this person can’t be convinced to do the right thing, they still want to say that, objectively, they are not acting in accordance with Reason. People do not imagine themselves as the ones in need of convincing, and they do not imagine finding themselves in a case where they wholly and unambiguously do not want to do the right thing. Rather, they think that the right thing to do will turn out to be something they prefer reasonably well, or would prefer if they had more self-control. The only role played by saying ethics is independent of desire is, therefore, to allow one to say others are wrong for not having one’s own preferences.

Your duty is to act on the principles that you’d rationally decide for everyone to be bound by. (My proof of this fact was too large to fit in the margins.) Sometimes, these will be principles that have you forego what you want most, because it’s better in general for agents to do so in such circumstances. If you haven’t, read about Yudkowsky’s True Prisoner’s Dilemma. It’s an important thing to internalize.

Now, let us imagine the 98%-Nazi-world again. In such a world, would it be permissible to commit random acts of violence to oppose what people do? No, of course not. Worlds where what’s allowed is determined by whoever is the most effective at violence are not good worlds. Worlds where I’m permitted to get what I want by any means are not worlds where I get what I want; by and large, they’re worlds where someone else does, and not necessarily someone who wants good things. It is important to note two things that hold about our hypothetical, insofar as they have any hope of being analogous to the case of animal-eaters: (i) people are not utterly insensitive to what the correct moral principles are; they do what they do out of self-deception, not a consistent decision to do whatever they want; (ii) 98% is a very large amount; given our supposition about most people in the hypothetical being so bad at wanting good things, this speaks massively against “Do whatever it takes when you’re in the 2%” being a good principle. The thing to do is convince others you’re right. If enough people are on your side that consideration (ii) isn’t so strong, then it becomes all the more feasible to stop others through legal means from doing the bad things they’re doing.3 In no case would random bombings or what have you be justifiable under normal circumstances.

And what of being friendly with others, in this hypothetical? There are a few things to address here. In some cases, maybe someone who says “If you were right about animal ethics, you wouldn’t be friends with carnists” might be making a claim of moral obligation.4 I don’t think that would be a plausible claim. A world where the minority of people who can see things clearly divest themselves from relationships with others is not a world where more people access important truths. Is friendship a tacit endorsement of someone’s behavior? Who cares? Nothing prevents you from untacitly disendorsing their behavior.

And these sorts of considerations are all the more clear-cut in the case of AI risk, where individual violence is so much more obviously unproductive, and where approaching the matter rationally is so essential. Worlds where it’s correct for the future to be decided by whichever side is best at taking matters into their own hands, again, can only by sheer accident be worlds where your goals are accomplished. It’s of little consequence that you are in fact right about the empirics, or that your goals are ones everyone is obligated to have. “Commit acts of violence only when you’re in fact on the right side” is the same principle as “commit acts of violence only when you think you’re on the right side.”