Null Results and File Drawers 🔸 The Cooptation of Science Reform 🔸 The Weaponization of Transparency

Bartosz Maćkiewicz and colleagues conducted a survey of 50 experimental philosophers in order to investigate the prevalence, causes, and potential solutions to publication bias and the file-drawer problem:

Contrary to the common assumption that journals reject null results, we found that most projects halt during early data analysis or presentation stages. The primary drivers for this attrition are resource constraints and competing priorities rather than statistical non-significance or rejection during peer review.

In their analysis, Maćkiewicz et al. found that

relatively few projects were discontinued at the submission or peer-review stages; when they were, the leading reasons concerned fit with a journal’s scope/profile and manuscript-related considerations, as well as judgments that results were unimportant. Notably, “null results” was among the least frequently selected reasons overall and was not selected at the submission or peer-review stages.

The researchers concluded that:

Overall, the results fit a multi-cause account of the file drawer problem and point to a diverse range of solutions.

Because a recurring theme is that studies remain unpublished for resource and prioritization reasons, remedies should minimize barriers.

Maćkiewicz, B., Kuś, K., & Krajewski, G. (2026). Publication bias in experimental philosophy: A survey on the file drawer problem, research attrition, and open science practices. Synthese, 208(2), 77. https://doi.org/10.1007/s11229-026-05727-2

In my last post, I discussed a special issue on feminist metascience. One of the articles in that special issue is by Molly King and Jeff Lockhart. It looked at women’s representation and participation in open access (OA) publishing. Based on a survey of 19,462 social scientists, they found “a distinct underrepresentation of women in green OA and on SocArXiv specifically.”

In a follow-up blog post, Lockhart discussed the cooptation of science reform ideas by anti-science actors and the incentives to sensationalize metascientific work in order to attract funding:

I don’t have a solution to the cooptation of science reform by anti-science actors. More sober language than “crisis” from the reformers would be good, but it would not prevent motivated actions like we have seen in the last few years. The language of ‘crises’ and brave ‘heterodox’ outsiders challenging consensus is an effective way to draw in some philanthropists and politicians. But as any good metascientist should recognise: there are incentives here. To attract ‘metascience venture capitalist’ money or excite the current federal science leadership, it pays to make bold, sensational claims and discredit past work.

Karen Levy and David Merritt Johns published a paper in 2016 titled “When open data is a Trojan Horse: The weaponization of transparency in science and governance.” As they explained:

Legislative efforts that invoke the language of data transparency can sometimes function as ‘Trojan Horses’ designed to advance goals that have little to do with good science or good governance.

Though transparency is often framed as an unalloyed good (provided that privacy interests can be adequately protected), in practice it provides a means through which diverse stakeholders attempt to achieve diverse political goals. Politically motivated proponents of transparency, in some cases, may exploit the epistemological and institutional realities that accompany the production of science and science-based policy.

What are these epistemological realities? They’re the inherent uncertainty of scientific conclusions:

Cacophony and contention are core elements of the scientific enterprise. By invoking a “narrow, idealized portrayal of science” in which research reliably produces clear and reproducible facts, the special interests behind the DAA, DQA, and SSRA mischaracterize science’s “inevitably incomplete, uncertain, contested, and … often unreliable” nature in their efforts to stymie regulatory activities with which they disagree.

Hence, bad actors may (a) mischaracterize science as producing clear and reproducible facts and then (b) weaponize transparency by using it strategically to manufacture doubt about undesirable “facts.”

Similarly, and in the same year, Stephan Lewandowsky and Dorothy Bishop warned that calls for open data may be motivated by “the aim of finding an analysis method that makes undesirable results go away,” and that “calls for more data can also be used to create the false impression that data are being withheld.” They strongly support open data but recommend protective action against the abuse of scientific transparency.

Share