Statistics Field Guide

Foundations

Read a diagnostic as evidence for a decision, not as a standalone score.

1. How to Read DiagnosticsWhat should a statistical diagnostic change about your analysis decision?

   Use withNull Hypotheses and P-valuesEffect SizesBootstrap and Permutation Baselines

2. Notation and EstimatorsWhat quantity are you trying to learn, what rule estimates it, and what did this particular dataset produce?

   Use withMeanVariance and Standard DeviationBootstrap and Permutation Baselines

3. Likelihood and Maximum LikelihoodWhich parameter values assign greater joint density or mass to the fixed observations under a declared model, and what does the maximizing value actually establish?

   Use withNotation and EstimatorsNull Hypotheses and P-valuesLikelihood-Ratio Test

4. Null Hypotheses and P-valuesHow incompatible is the observed statistic with a prespecified null model, and what decision may that incompatibility support?

   Use withAugmented Dickey-FullerKPSSKolmogorov-Smirnov Distance

5. Likelihood-Ratio TestHow much maximized fit is lost when the null restriction is imposed?

   Use withLikelihood and Maximum LikelihoodNull Hypotheses and P-valuesWald Test

6. Wald TestHow far is the unrestricted estimate from the null value on its estimated local uncertainty scale?

   Use withLikelihood and Maximum LikelihoodNull Hypotheses and P-valuesLikelihood-Ratio Test

7. Score TestDoes the likelihood still point away from the null when evaluated at the best null-constrained fit?

   Use withLikelihood and Maximum LikelihoodNull Hypotheses and P-valuesLikelihood-Ratio Test

8. Likelihood-Based Test ComparisonWhich likelihood-based test matches the fits and evidence available, and why can likelihood-ratio, Wald, and score results separate in finite samples?

   Use withLikelihood and Maximum LikelihoodNull Hypotheses and P-valuesLikelihood-Ratio Test

9. Effect SizesHow large is the estimated contrast, on which scale, and relative to which decision threshold?

   Use withCohen's dWasserstein DistanceKolmogorov-Smirnov Distance

10. Bootstrap and Permutation BaselinesHow variable is this estimator across plausible resamples, or how unusual is this statistic after legally breaking the tested relationship?

    Use withMutual InformationDistance CorrelationSilhouette Score