What is statcheck?

statcheck is a “spellchecker” for statistics. It checks whether your p-values match their accompanying test statistic and degrees of freedom.

statcheck searches for null-hypothesis significance test (NHST) in APA style (e.g., t(28) = 2.2, p < .05). It recalculates the p-value using the reported test statistic and degrees of freedom. If the reported and computed p-values don’t match, statcheck will flag the result as an error.

What can I use statcheck for?

statcheck is mainly useful for:

1. Self-checks: you can use statcheck to make sure your manuscript doesn’t contain copy-paste errors or other inconsistencies before you submit it to a journal.
2. Peer review: editors and reviewers can use statcheck to check submitted manuscripts for statistical inconsistencies. They can ask authors for a correction or clarification before publishing a manuscript.
3. Research: statcheck can be used to automatically extract statistical test results from articles that can then be analyzed. You can for instance investigate whether you can predict statistical inconsistencies (see e.g., Nuijten et al., 2017 doi:10.1525/collabra.102), or use it to analyze p-value distributions (see e.g., Hartgerink et al., 2016 doi:10.7717/peerj.1935).

How does statcheck work?

The algorithm behind statcheck consists of four basic steps:

1. Convert pdf and html articles to plain text files.
2. Search the text for instances of NHST results. Specifically, statcheck can recognize t-tests, F-tests, correlations, z-tests,