I recently travelled to Halifax, Nova Scotia for the 2026 Canadian Philosophical Association (CPA) Congress, where I presented some co-authored work from the PolyGraphs project (an investigation of the effects of mis- and disinformation on people’s attitudes).1 I was delighted to get some helpful commentary from Arianna Falbo, Assistant Professor of Philosophy at Toronto Metropolitan University, whose remarks seemed to me to raise concerns about the idealizations involved in our computational methodology that others may share.2 What follows is an abridged version of Arianna’s comments, followed by some reflections they prompted, in the hope that these may be of interest beyond the audience present at the CPA talk, and independently of some of the specifics of the paper presented.
Arianna began with an…
Important Disclosure: I’m not very familiar with the literature that you are engaging with, nor with the methodology of using simulations to explore how beliefs spread in social networks…for that reason some of my comments will concern some more big-picture questions about the methodology and what we’re meant to extrapolate from the results outlined in the paper.
Despite her modesty, Arianna’s comments were very much on point. She began by noting that…
One key conclusion of the paper is that discovering truth in communities depends not only on information quality, but also on social structure: how people are related to each other, how easily they can process and share information. Crucially: better-connected communities are not necessarily better-informed communities -- under conditions of misinformation or disinformation, increased connectivity can sometimes worsen epistemic outcomes. These insights further support and expand upon the “Zollman effect” (due to Kevin Zollman, 2007) which has demonstrated a trade-off between the reliability of a community (how likely it is to converge on the truth) and the speed with which it reaches a consensus.
It’s natural to think that it would be good to widely share one’s initial findings or preliminary results with the broader research community—it seems like this will help the community converge on the truth faster. But it turns out that in some cases this has the opposite effect: the Zollman effect demonstrates that sharing initial evidence sooner, and with a greater number of researchers, can sometimes mean sacrificing accuracy for the group overall. This can happen when the group is more likely to converge prematurely on early misleading evidence, when that evidence is widely shared, which may cause the group to overall abandon otherwise promising theories or lines of inquiry.
Here is an example [Brian’s note: very Canadian in its subject matter!] which, I think, exhibits (something like) the Zollman effect in every-day life:
Imagine a new poutine place, Gravy Train, opens in your town and it serves poutine that is slightly better than the poutine served at the popular chain that everyone already knows and loves, Smokes. Your friend group is wondering if the new poutine place is any good. A couple of your friends try the new poutine at Gravy Train, but they happen to visit on a bad day—it was during a very busy time, and their poutine got less cheese curds than normal because the cook was having a bad day (this is noisy/misleading initial evidence). Your friends think the meal was lackluster. They post a message in your friends group chat, it reads: “Just tried Gravy Train’s poutine. Not great.” (Rapid information spread to others in the group.) As a result, the group updates on this information, and they prematurely converge on the view that Smoke’s is better than Gravy Train. But had they never received this information, others would have checked out Gravy Train themselves (conducting “independent trials”) and discovered that, on average, it’s slightly better than Smoke’s. Gravy Train would have had time to prove itself as a superior option.
In this case, fast-spreading early information can shut down exploration and opportunities to discover the true (slightly better) option. This is a tragic result – it leads to us having poutine that is inferior. As the authors nicely put it in the paper: “Connectivity can be bad for accuracy!”
Having provided this context, Arianna then raised the following issue for discussion:
What can we extrapolate from the data about how we should think about the epistemic dynamics of social groups who are engaged in collective inquiry or problem solving? One might worry that the methodology is perhaps too idealized to be helpful in providing concrete or action-guiding advice.
Here is one idealization in the methodology of the paper that I was feeling a bit suspicious of - it concerns how misinformation is represented in the simulations. Specifically, the paper models misinformants as “picking what to report at random (e.g., by tossing a fair coin).” This makes things easy to operationalize in the simulated models, but I worry that this also oversimplifies what misinformation really is, and the more complex ways it can spread within social groups.
There are different ways misinformation might spread. (1) Misinformation might spread when someone shares information which is straight up false. Or, perhaps (2) we can share misinformation when we share information that is organized or curated or presented in a way that’s misleading, e.g., how a narrative or newspaper or search engine or algorithm may organize content in a way that is misleading because it makes some information more salient, thus suggests that it is more important or more relevant to some issue or more worthy of attention, etc., when it is not. Or maybe (3) misinformation can spread via the sharing of true statements, which lack crucially important context. Here is an example from O’Connor and Weatherall (2024): someone might report that “the well-respected medical journal Lancet published an article linking autism and vaccines,” This is true, but it’s missing crucially important context, namely, that the article was retracted by the journal in 2010, that hundreds of follow-up studies failed to replicate its findings, and that the overwhelming consensus among medical researchers is that there is no such link.
In short:
One might worry that the way the simulations represent misinformation is too idealised and it perhaps oversimplifies or flattens important differences concerning how misinformation spreads in practice, and how it obstructs the process of belief formation and knowledge acquisition.
The central worry Arianna raises, and which others may share, is that my co-authors and I treat misinformation as false content that is generated through a neutral binomial probabilistic distribution, and that this may not be what it is in all cases. In response, I want to make two replies: one substantive, and one methodological.
There are perhaps two aspects to the concern Arianna raises: one is the question of why this particular distribution. Why not e.g. a neutral, but random uniform distribution? Or perhaps some other, biased distribution? Might it not turn out that, for e.g. ideological reasons, there is bias even in false content that is disseminated without intent to deceive?
The other is the question of whether misinformation must be false. Isn’t it enough if it is misleading? Here, on this substantive point, I want to put my foot down – though with a rationale. I think some are inclined to allow true but misleading content as misinformation because it can misinform. There is a kind of Aristotelian residue to the (ordinary, pre-Shannon) idea of information: becoming informed is taking the form of the thing (but not its matter) into one’s mind. And I can certainly end up believing something false (becoming inaccurately informed) based on (the possibly misleading presentation of) something true – this is, in a way, what Zollman showed, and what Arianna points to with the example of the study linking vaccination and autism in her response. Moreover, if I have become misinformed, then there was, presumably, some act of misinforming me – and therefore some misinformative activity, some misinformation that occurred. But that is not to say that the content shared with me was itself misinformation. Philosophers are well aware of the act-object distinction, and it seems to me that it comes to the defence of the idea that misinformation itself is necessarily false content. (My belief, of course, has false content – what I believe, the object of my belief, is false; but it doesn’t follow that the content, or object, of the communicative act that led to it was false/misinformation.)
My thought, specifically on the first aspect of the issue of what misinformation is, when I co-authored the first (2024) paper on this topic was that we needed an operationalization of the notion of misinformation if we were to do research – and one way that someone might spread false information unintentionally is through incompetence. They aren’t trying to deceive, they just aren’t very good at believing the truth. One way they might be incompetent is that they might, e.g., undertake an experiment – in our case, a series of (biased) coin tosses – but then forget the outcomes. Maybe they didn’t write them down. So when it came to them reporting their observations, they tried to recall each coin toss outcome in turn, but were essentially guessing at random, writing down those guesses and reporting their aggregate.
Is this the only way in which one might disseminate misinformation? Of course not. But it is one concrete way in which someone might purvey misinformation without intending to do so.
Philosophers are inclined to worry about the provision of analytically correct definitional accounts of various notions – but those wishing to generate knowledge of synthetic truths need to gather relevant evidence and then talk about it. So as I think of the operationalization of the notion of misinformation used, it is not to be understood as some kind of context-insensitive, universally applicable, definition; rather, it is just a perfectly good, contextualized understanding that allows us to make some progress in modelling, and thereby producing knowledge about, a phenomenon of interest. And by all means, we should also investigate other operationalizations - no doubt there’s an experiment to be done exploring such alternatives!
So I guess this response to Arianna’s methodological point is more concessive: sure, what my co-authors and I say about misinformation in the paper may not generalize beyond the context that makes the operationalization appropriate. But that just means that care is needed in reporting the findings - and that further research is needed to gain yet more knowledge on this important topic.
For more on Polygraphs, see this previous post on The Computational Philosopher.
In fact, the paper has enough co-authors that some attendees asked me if my session was a symposium!
Many thanks to Arianna for her permission to share some of her comments.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.