How to Identify Reliable Facts in a Broken Media Ecosystem

I recently argued that it is currently possible to cut through the pseudo-news that dominates the on-line and cable news environments by means of AI. Here we’ll look at the basic strategy for this, which is the same whether you use AI or not for this purpose. At present, OpenAI’s ChatGPT and Anthropic’s Claude can be relied upon to support these strategies, and even to employ them automatically in their basic news search functionality. But even if one is only searching in the old fashioned way for reliable information about current events—Google searches, e.g.—these same strategies and principles still apply.

Political polarization is now a familiar fact of American life, but it is reinforced by a less obvious development: a fragmented media environment that lets each of us shop for the news, commentary, and “facts” we like believing. As a result, we do not merely have two political parties; we have, in effect, two competing realities. Yet there can be only one reality, and the existence of these divergent versions is evidence of serious error. Perhaps both sides are distorted, but at least one of them must be.

I take it as axiomatic that if two news sources offer contrary facts about the same topic, at least one of them is simply wrong.

I also maintain that, with modest effort, one can usually determine whether a factual claim is trustworthy.

I recognize that factuality as a category of knowledge is contestable at a philosophical level, and that postmodern theory in particular has long challenged this idea, arguing that “facts” are inseparable from interpretive frameworks or power structures. But the popularized versions of that critique—the stance that all claims are equally valid and that “You believe what you like; I’ll believe what I like”—is not a sophisticated epistemology. It is a casual nihilism masquerading as open-mindedness, a denial that empirical evidence must compel our assent if it doesn’t fit our narrative. Some things are simply true, whether or not they flatter our “worldview.”1)

It’s an old question; how do we know what we don’t know? Plato depicted Socrates considering the basic contours of this vexing epistemological quandary two and a half millennia ago in the Meno. It is a question that—merely as a sincerely felt question—is already indicative of a certain level of self-awareness and intellectual virtue, irrespective of any answer. It is easy to embrace a lovely speech or a piece of writing that confirms what one already believes to be true. Serious people are obliged, however, to interrogate and challenge their own views as well as those of others. Serious people are keen to become aware of their own ignorance, to learn what they don’t know. It is only a question of how interested we are in being serious.

There are many forms this kind of intellectual seriousness can take; what I’m interested in exploring here is the kind of standards we should apply to journalistic news and analysis. If we point to a piece of writing as evidence for our opinion, what level of scrutiny should we apply to that piece of writing before placing our weight on it? Is it enough that a piece of writing harmonizes with our long-held opinions? Or do we acknowledge that the value of the piece depends on the veracity of its fact claims and the strength of is reasoning?

If we do think facts and sound reasoning are important, we should evaluate speeches and writings on this basis. Evaluating sound reasoning is something each of us in principle can do on our own. If we have even a pedestrian familiarity with basic logic and critical thinking, we can discern a spurious argument from a sound one. Facts might pose a greater difficulty. How can we know whether a fact claim is true? This necessarily involves checking a claim against authoritative sources.

But what constitutes authority? This is where the controversy usually enters. Who is the authority, for example, on the safety and efficacy of a particular vaccine: an epidemiologist or a layman with very strong views, who cites a series of anecdotes that seem to vindicate his views? If these two sources contradict each other, and if we feel that it’s important to listen to “both sides” and to attribute equal authority to both voices, then how do we decide which one is right? Do we just go with what we like better, what feels right to us? Or is there some standard of evidence that renders these two opinions unequal in authority?

What if 95% of epidemiologists hold one view, while a group of non-specialist medical professionals holds a contrary view? Do we still maintain that there is no basis for final authority, other than personal inclination?

If we feel that there is no basis for granting authority to any particular voice in such a dispute, then we live in a world where there is no truth other than one’s own inclinations. This is an easygoing nihilism.

Share

A postulate I hold here is that there is such a thing as expertise. Study and institutional rigor generate insights into the world that hold a special status. This does not imply that expert opinion is simply true and unquestionable. It only means that it holds a status that is more reliable than non-expert views on certain topics.

This distinction makes possible a basis for navigating and adjudicating claims of fact. It does not give us absolute truth, but it does give us something better than nihilistic relativism. It is on this basis that we have massive libraries of scientific papers that hold authority until other papers prove them wrong. What results is a higher echelon of published research that survives refutation and gains the status of scientific consensus. This should be taken seriously.

This is true not only for medical questions and other empirical questions of natural science, but also for any topic where people actually measure and count things. It is true for all matters of fact. To be sure, facts are not the whole story. Values are also very important, and reasonable people disagree about what’s more and less valuable. But what we’re talking about here are facts, and with facts expertise matters.

If we agree about this, then the next step is to compare claims of fact to expert sources. This is a long and tedious process and while some of us have the time and ability to do this with some fact claims, almost no one can do it with all the claims that are out there. The good news is that there are organizations who make it their business to check facts at scale. These organizations take on the work of correlating scientific and institutional record keeping with journalistic fact claims. Many of these organizations also look at argumentation and context to evaluate not just the truth of a claim, but also the integrity of the arguments that are made on the basis of the fact claims. Sometimes a fact claim can be true, while the use made of the fact is misleading.

If there were only one or two such organizations it would be difficult to be sure whom to trust without replicating their work. Fortunately, there are many such organizations, and their results are not hard to access. When their documented analyses converge it is less easy to dismiss the results.

https://www.factcheck.org

https://www.politifact.com

https://www.snopes.com

https://www.leadstories.com

https://www.usatoday.com/news/factcheck

https://apnews.com/hub/ap-fact-check

https://www.reuters.com/fact-check

https://www.sciencefeedback.co

https://factly.in/category/english/

https://climatefeedback.org

https://healthfeedback.org

https://toolbox.google.com/factcheck/explorer

https://adfontesmedia.com

https://www.newsguardtech.com

https://mediabiasfactcheck.com

https://ground.news

https://allsides.com

https://chartbeat.com

https://presswatchers.org

You might reasonably ask, who’s checking the fact checkers? There are many academic media observatories that evaluate these fact checking tools for their reliability.

https://shorensteincenter.org – Shorenstein Center (Harvard University)

https://comprop.oii.ox.ac.uk – Computational Propaganda Project (Oxford Internet Institute)

https://reutersinstitute.politics.ox.ac.uk – Reuters Institute for the Study of Journalism (University of Oxford)

https://mediaengagement.org – Center for Media Engagement (UT Austin)

https://www.poynter.org/news/fact-checking – Poynter Institute / International Fact-Checking Network

https://reporterslab.org – Duke Reporters’ Lab

https://www.annenberg public policy center.org – Annenberg Public Policy Center (University of Pennsylvania)

This is, admittedly, a daunting list. It would be unrealistic to suggest that everyone who wishes to be serious about politics and media grapple with all these resources. Yet citizens of a free society bear some responsibility to ensure that their convictions rest on something firmer than instinct or habit. We all have blind spots, and we should make a disciplined effort to discover what they are. One of the resources I listed above is a good one-stop shop for basic media awareness: Ground News. It’s designed to highlight blind spots on both the left and the right. If you don’t already have a routine where you read widely from multiple perspectives, using fact checking resources regularly, I recommend Ground News as a way to cultivate a sensitivity to these concerns. (I promise I’m not getting paid for this plug.)