Rena Alcalay and Joachim Vandekerckhove discuss “epistemic disadvantage in open science practices.” Following Davis-Stober et al. (2025), they distinguish between two attitudes to the scientific literature: the Book of Truths (BoT) and the Book of Conversations (BoC):
Those holding a BoT attitude see the primary purpose of the academic literature as creating a definitive, reliable repository of established knowledge, a canon that can be assessed and trusted at face value by expert and lay readers alike. By contrast, those adopting a BoC attitude view academic literature primarily as an ongoing exchange of ideas and communication among researchers, valuing provisional findings, preliminary data, and emergent questions as inherently worthwhile contributions to cumulative science, regardless of whether they achieve formal publication or definitive verification.
These divergent attitudes imply fundamentally different thresholds for what counts as acceptable epistemic risk in data sharing. Where a BoC attitude treats intermediate, unvetted data as intrinsically valuable, a BoT attitude worries that sharing raw findings risks misleading the scientific community or distorting the epistemic record. What appears to one researcher as reckless exposure to another appears as essential democratisation. This attitudinal divide, we argue, underlies many tensions in contemporary OS [open science] practice.
Andy Stirling wrote a Substack post on the way in which metascience addresses uncertainty, directionality, and democracy. I’ll pull out three paragraphs on directionality here, but the whole post is worth a read:
We often treat key foci of metascience as if they are simply about magnitudes, whose main relevance is whether they grow or shrink in scale – more or less. This is true, for instance, of ‘excellence’, ‘novelty’, ‘impact’, ‘productivity’, ‘pace’ – or ‘winning’ or ‘losing’ in some particular research or innovation ‘race’. All are treated as if they are simple self-evident magnitudes – independent of any judgement about how to measure, by what scales, aggregated how and with what relative weightings?
In other words, these major subjects of metascience are treated as ‘scalars’ (simple single numbers) rather than as what they really are: vectors (with multiple constituting dimensions that give them directions as well as scale). All the above values, for example, rest not just on ‘how much?’, ‘how fast?’ or ‘who wins?’ but on fundamentally qualitative alternative directions for possible change. Inconvenient to interests that wish to conceal awkward resulting questions, a crucial truth in metascience is that the most defining properties of any given field of research or innovation are given not simply by associated magnitudes, but by their directions.
This suggests a subtle but profound shift for metascience. Evaluation should become less concerned with demonstrating what one particular pathway can achieve, and more concerned with comparing alternative pathways. Objectivity lies not in presenting a single notionally authoritative picture, but in exploring systematically how different assumptions, values and priorities generate different pictures.
Madeleine Pownall published an article celebrating the historical contributions of women as champions of open science principles:
Contemporary narratives often frame Open Science and metascience as novel frontiers of transparency and rigor, obscuring their deep roots in feminist methodologies and critical social science traditions. Rather than asking how feminist and qualitative research can conform to Open Science standards, we should recenter the values of reflexivity, care, inclusivity, and accountability long championed by feminist scholars (Siegel et al. 2021). The methodological critiques and calls for accountability articulated by earlier women scholars anticipated many reforms now gathered under Open Science; had these insights been integrated, the severity of today’s “crisis” might have been reduced.
Pownall, M. (2026). Women as the first open scientists: Five stories of the neglected contributions of women in (social) science reform history. Gender & Society. https://doi.org/10.1177/08912432261466293
Back in 2016, Hans Phaf addressed the distinction between exploratory and confirmatory research offered by Wagenmakers et al. (2012):
The primary aim of this comment is to juxtapose the statistically oriented approach and a more theoretically oriented approach. The statistical approach of Wagenmakers et al. (2012) entails a two-way classification in either exploratory or hypothesis-confirmatory research. The latter type can only have a binary outcome with respect to the decision being made, the hypothesis is either confirmed or not. To arrive at such an outcome, a replication attempt must rely on the original research having uncovered and made explicit all relevant processes (i.e., an exhaustive theoretical analysis). All other types of research fall in the exploratory category, even when they further develop the theory starting from quite specific hypotheses. Merely confirming preregistered hypotheses has, however, never yielded new hypotheses, whereas unexpected findings stimulating further investigations do have that capacity and may even be the royal road to scientific innovation (e.g., Lehrer, 2009). Calling it undirected exploratory research, moreover, also does not do justice to the gradual progress-by-adjustment type of research (cf, Lakatos, 1970). The latter type of research is often guided by well-specified and concrete process hypotheses, which may be far superior above merely expecting a difference.
Phaf, R. H. (2016). Replication requires psychological rather than statistical hypotheses: The case of eye movements enhancing word recollection. Frontiers in Psychology 7. https://doi.org/10.3389/fpsyg.2016.02023
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.