Bayes’ Theorem from Uncertainty to Decision
bayes-theorem, grid-approximation, posterior-sampling, normal-distribution, data-visualisation
A probability model is a disciplined way to reason when one answer is unknown. It does not remove uncertainty. It records candidate answers, states how observations would arise under each candidate, and updates their relative plausibility when data arrive.
This tutorial rebuilds three simulations I first made with Daniel Slutsky’s JointProb group. The examples come from sections 2.2, 3.2, and 4.3 of Richard McElreath’s Statistical Rethinking. No statistics or programming background is assumed. Each chapter asks one practical question:
- Update
How should observations change uncertainty? - Sample and decide
How can a distribution guide an action? - Scale up
How does the same update work with two unknown parameters? - Reproduce
How do code, seeds, tests, and rendering make the lesson checkable?
The five ideas under every update
Suppose I do not know what proportion of a globe is covered by water. Uncertainty means more than one answer remains credible. A probability is a number from 0 to 1 used here to describe how strongly the model supports an event or candidate. The unknown water proportion, called \(p\), is a parameter: a quantity the model tries to learn. A model is the set of assumptions connecting that parameter to possible observations. The observed water and land outcomes are the data.
- Uncertainty
- Several answers are still possible.
- Probability
- A 0-to-1 numerical description of uncertainty inside a stated model.
- Parameter
- An unknown quantity in the model, such as the water proportion p.
- Model
- Assumptions saying how data could be generated for each parameter value.
- Data
- Recorded observations used to compare the candidates.
I cannot calculate with every decimal between 0 and 1 directly in this simple demonstration, so I use grid approximation: replace the continuous range by 201 equally spaced candidate values.
\[p \in \{0, 0.005, 0.010, \ldots, 0.995, 1\}.\]
Read this as: “\(p\) is one of 201 candidates from zero to one, spaced by 0.005.” A finer grid would approximate the continuous range more closely but would require more calculation.
p
A candidate value for the unknown proportion of the globe covered by water; it must lie between 0 and 1.
∈
‘Is an element of’ or ‘is one of the values in.’
{…}
Braces list the allowed candidate values rather than one continuous interval.
0.005
The distance between adjacent candidates. Including both endpoints gives 201 grid points.
About this equation
The grid is a numerical approximation; the underlying proportion is still conceived as continuous.
Code detail: Constructing the 201-point probability grid
Source: bayes_theorem_simulations.clj — probability-grid
probability-grid
The code name for the ordered collection of candidate values represented by p in the equation.
range
Produces the integer numerators 0 through 200; dividing by 200.0 creates the 0.005 spacing.
Clojure's range produces the integers 0 through 200. Dividing each by 200 gives 0, 0.005, …, 1. The result is stored as a vector so its order is stable.
(def probability-grid
(mapv #(/ % 200.0) (range 201)))Check: (count probability-grid) must equal 201, and the assertion at the end of this article enforces it.
Before data, each candidate receives a prior weight. After data, the likelihood says how compatible those data are with each candidate. Their product is an unnormalised weight: useful for ranking, but not yet a set of probabilities that sums to 1. Normalisation divides every product by their total. The resulting posterior is the updated distribution over candidates.
1. Update: learning from globe tosses
Imagine tossing and catching a globe. The point under a finger is recorded as water or land. If a candidate says water is common, water observations should be less surprising under it. This is the likelihood’s job.
For \(W\) water observations and \(L\) land observations, one particular ordered sequence has probability
\[p^W(1-p)^L.\]
In plain English: multiply \(p\) once for every water result and \(1-p\) once for every land result. This assumes the observations are independent once \(p\) is fixed.
p
The probability that one independent toss lands on water.
1 − p
The complementary probability that one toss lands on land.
W
The number of water observations in the sequence.
L
The number of land observations in the sequence.
p^W(1 − p)^L
The product of all water and land probability contributions.
About this equation
This expression describes one specified ordering, such as W–L–W.
Code detail: Probability of one ordered water–land sequence
Source: bayes_theorem_simulations.clj — ordered-sequence-probability
p
The candidate water probability p from the equation.
water
The code name for W, the number of water observations.
land
The code name for L, the number of land observations.
(defn ordered-sequence-probability [p water land]
(* (Math/pow p water)
(Math/pow (- 1.0 p) land)))The implementation uses descriptive count names while preserving p exactly.
When only the counts matter, there are
\[\binom{W+L}{W}=\frac{(W+L)!}{W!L!}\]
Code detail: Counting water–land orderings
Source: bayes_theorem_simulations.clj — binomial-coefficient
n
The code name for W + L, the total number of observations.
k
The code name for W, the positions chosen for water observations.
acc
The running product used to compute the coefficient without expanding three factorials.
(defn binomial-coefficient [n k]
(let [k (min k (- n k))]
(reduce (fn [acc i]
(* acc (/ (- (inc n) i) i)))
1.0
(range 1 (inc k)))))This product is algebraically equivalent to the factorial ratio in the equation.
possible orderings. Read this as: choose which \(W\) of the \(W+L\) positions contain water. The exclamation mark means factorial—for example, \(3!=3\times2\times1\). Multiplying the ordering count by the probability of one ordering gives the binomial likelihood:
\[\Pr(W,L\mid p)=\binom{W+L}{W}p^W(1-p)^L.\]
Read this as: “if candidate \(p\) were fixed, what probability would this model assign to seeing these water and land counts?” Once the data are fixed and candidates vary, the same expression is called a likelihood.
Pr(W,L | p)
The probability of the observed counts, conditional on candidate p.
|
‘Given’ or ‘conditional on.’
(W + L choose W)
The number of distinct sequences with the same counts.
n!
n factorial: multiply the positive integers from n down to 1.
p^W(1 − p)^L
The probability of one particular ordering.
About this equation
The ordering count is constant across p for fixed data, so it changes scale but not the likelihood curve's shape.
Code detail: Evaluating the binomial likelihood
Source: bayes_theorem_simulations.clj — binomial-likelihood
p
The candidate water probability p_i evaluated by the grid.
water
The code name for W in the likelihood equation.
land
The code name for L in the likelihood equation.
binomial-coefficient
Computes the number of orderings with the observed counts.
ordered-sequence-probability
Computes p^W(1-p)^L for one ordering.
(defn binomial-likelihood [p water land]
(* (binomial-coefficient (+ water land) water)
(ordered-sequence-probability p water land)))The two named factors correspond to the two factors in the displayed likelihood.
CheckThe executable six-water, three-land example uses 201 candidates and peaks at p = 0.665.
Bayes’ theorem names this update:
\[\Pr(p\mid W,L)=\frac{\Pr(W,L\mid p)\Pr(p)}{\Pr(W,L)}.\]
Read it from right to left: multiply the prior support for candidate \(p\) by the likelihood of the observations under that candidate, then divide by the overall probability of the observations so all posterior probabilities sum to 1. On this finite grid, normalising all products performs that division.
Pr(p | W,L)
The posterior support for p after observing W water and L land outcomes.
Pr(W,L | p)
The likelihood of those observations under candidate p.
Pr(p)
The prior support for p before these observations.
Pr(W,L)
The overall or marginal probability of the data; it normalises the products.
likelihood × prior
The unnormalised posterior weight for candidate p.
About this equation
On a finite grid, divide each likelihood-times-prior product by the sum of all products.
Code detail: Normalising likelihood-times-prior weights
Source: bayes_theorem_simulations.clj — grid-posterior
prior
The vector containing Pr(p) for every candidate p.
water
The code name for W, the observed water count.
land
The code name for L, the observed land count.
binomial-likelihood
Computes Pr(W,L | p) for each candidate.
normalize-mean-one
Applies the common normalising scale represented by Pr(W,L).
The pipeline evaluates the likelihood at every candidate, multiplies candidate by candidate with the prior, then applies one common scale factor. Scaling to mean 1 rather than sum 1 preserves the same posterior shape used by the charts.
(defn grid-posterior [prior water land]
(->> probability-grid
(mapv #(binomial-likelihood % water land))
(mapv * prior)
normalize-mean-one))Check: the article asserts that the 201 weights' arithmetic mean is 1 to floating-point tolerance.
A sequential update needs no new rule. After one observation, the posterior contains everything this model carries forward about \(p\). It therefore becomes the next observation’s prior: yesterday’s posterior is today’s prior. The dynamic strip inside the simulator shows the current prior, latest-observation likelihood, raw product, and normalised posterior.
Predict before revealing: what should one water observation do?
First make a prediction. Should candidates near p = 0, p = 0.5, or p = 1 gain the most relative support?
Reveal: one water observation has likelihood p, so it favours larger p values. A land observation has likelihood 1 − p and favours smaller values.
How to read the globe simulator
- Horizontal position is candidate water proportion p, from 0 to 1.
- Curve height is relative support or likelihood; it is not the probability of one exact p.
- Use Water or Land for deliberate evidence. Random sample uses hidden true p = 0.6.
- Change the prior to see its influence early; add data to see the likelihood increasingly dominate.
The two full-likelihood charts differ by a constant multiplier, so their normalised shapes match.
Loading the globe-toss Bayesian update simulator…
Code detail: Seeded random globe observations and reset
A random seed is a starting number for a repeatable pseudo-random sequence. Reset creates the generator from the same seed again, so it replays the same simulated observations. That is deterministic replay: identical versioned inputs and seed produce identical results.
(def update-seed 20260713)
(defn generated-observation! []
(if (< (uniform! @update-rng) 0.6) :water :land))
(defn reset-update! []
(cancel-update-timer!)
(reset! update-rng (make-rng update-seed))
(let [prior (:prior @update-state)
speed (:speed @update-state)]
(reset! update-state
(assoc initial-update-state :prior prior :speed speed))))The browser keeps the sequence already drawn in browser state—data held in memory for the current page. Clearing replaces that state and rewinds the generator.
Build: list candidate values and specify a prior and likelihood. Check: normalise and inspect the update after known observations. Decide: carry the posterior forward as the prior for the next observation.
2. Sample: turning uncertainty into a decision
A posterior is a distribution, not automatically a single estimate. To act, we must say what action is available and what mistakes cost. A decision is the chosen action. A loss function assigns a cost to each possible decision–truth pair. Change the loss and the best estimate can change even when the posterior does not.
Here the decision \(d\) is a claimed water proportion. A perfect answer earns $100; every 0.01 of absolute error loses $1. The expected absolute loss is
\[E[|d-p|\mid W,L] = \sum_p |d-p|\Pr(p\mid W,L).\]
Read this as: for each possible \(p\), calculate how far decision \(d\) misses, weight that error by the posterior support for \(p\), and add. Choose the \(d\) with the smallest weighted average error.
d
A candidate decision: the water proportion chosen for the bet.
p
One possible true water proportion on the grid.
|d − p|
Absolute error: the non-negative distance between decision and possible truth.
E[… | W,L]
Expected value after conditioning on the observed counts.
Σ_p
Add one weighted error for every candidate p.
Pr(p | W,L)
The posterior probability weight assigned to candidate p.
About this equation
The decision with the smallest posterior-weighted average absolute error is preferred.
Code detail: Computing posterior expected absolute loss
Source: bayes_theorem_simulations.clj — expected-absolute-loss
posterior
The code collection containing the weights Pr(p | W,L).
d
The candidate decision d whose expected loss is evaluated.
p
One candidate water proportion p from probability-grid.
weight
The posterior weight paired with the current p.
weight-total
The normaliser; the article stores mean-one rather than sum-one weights.
(defn expected-absolute-loss [posterior d]
(let [weight-total (reduce + posterior)]
(/ (reduce + (map (fn [p weight]
(* (Math/abs (- d p)) weight))
probability-grid posterior))
weight-total)))The map multiplies each absolute error by its posterior weight; the reduction implements the sum.
CheckFor the executable six-water, three-land example, the grid decision minimising expected absolute loss is d = 0.645.
Directly evaluating every \(d\) works on this small grid. Another route is posterior sampling: draw candidate values with frequency proportional to their posterior probability. The cloud of draws approximates the distribution. Under absolute loss, the posterior median minimises expected loss.
Repeating random draws to approximate a distribution or numerical result is Monte Carlo approximation. More draws reduce simulation noise, but do not repair a poor model. This interaction animates 2,000 draws and compares them with a fixed 10,000-draw run.
Predict before revealing: where should the sample median settle?
Inspect the posterior from 100 globe observations. Predict whether its median will settle below, near, or above the hidden true p = 0.6.
Then test it: start slowly enough to see individual draws, increase the speed, and compare the live median with the direct loss minimum.
How to read the sampling simulator
- The first row fixes one 100-observation dataset and shows its posterior and loss curve.
- The trace plots draw order vertically by sampled p; the histogram counts draws; the density rescales those counts.
- The animated and fixed panels use different seeded streams but the same posterior.
- Watch the running median wobble, then stabilise as the number of draws increases.
Loading the posterior sampling and decision simulator…
Code detail: Drawing grid values in proportion to posterior weight
First normalise the 201 weights so they sum to 1. Draw a uniform number between 0 and 1, then walk through cumulative posterior mass until it reaches that number. Return the corresponding grid value.
(defn weighted-grid-sample! [rng]
(let [target (uniform! rng)]
(loop [index 0
cumulative (first fixed-posterior-mass)]
(if (or (>= cumulative target)
(= index (dec (count probability-grid))))
(nth probability-grid index)
(recur (inc index)
(+ cumulative
(nth fixed-posterior-mass (inc index))))))))The fixed 10,000 draws are computed once from their own seed. The animated stream is reset independently, so interaction never changes the comparison result.
Build: define the action and its loss. Check: compare the direct grid minimum with a seeded Monte Carlo approximation. Decide: report the estimate appropriate to the declared loss, not a context-free ‘best’ number.
3. Scale up: a Gaussian model with two parameters
The globe model had one parameter. Adult height introduces two. A Gaussian distribution—the familiar symmetric bell shape—is described by its mean \(\mu\), which locates the centre, and standard deviation \(\sigma\), which measures typical spread around the centre. Its density describes relative concentration near a height; because height is continuous, density at one exact point is not itself a probability.
For observed height \(h\), the Gaussian density under candidate \((\mu,\sigma)\) is
\[f(h\mid\mu,\sigma)= \frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{(h-\mu)^2}{2\sigma^2}\right).\]
Read it as: density is highest when \(h\) is near candidate mean \(\mu\) and falls as the squared distance grows; candidate spread \(\sigma\) controls how quickly it falls. For a fixed observed height, comparing this density across parameter pairs gives the likelihood.
f(h | μ,σ)
The density at height h for the candidate Gaussian described by μ and σ.
h
The observed adult height.
μ
The candidate mean, locating the distribution's centre.
σ
The positive candidate standard deviation, controlling spread.
π
Pi, appearing in the Gaussian normalising factor.
exp(a)
The exponential function e raised to power a.
(h − μ)^2
The squared distance of the observation from the candidate mean.
About this equation
For fixed h, this density is the observation's likelihood across candidate μ and σ pairs.
Code detail: Evaluating the Gaussian density
Source: bayes_theorem_simulations.clj — normal-density
x
The implementation name for observed height h.
mean
The implementation name for candidate mean μ.
sd
The implementation name for candidate standard deviation σ.
Math/PI
The implementation constant for π.
Math/exp
The exponential function exp used by the density.
(defn normal-density [x mean sd]
(/ (Math/exp (/ (* -0.5 (Math/pow (- x mean) 2.0))
(Math/pow sd 2.0)))
(* (Math/sqrt (* 2.0 Math/PI)) sd)))The names x, mean, and sd are the executable counterparts of h, μ, and σ.
The preserved grid contains 41 means from 150 to 160 cm crossed with 41 standard deviations from 7 to 9 cm: \(41\times41=1{,}681\) candidate models. Each observed height supplies a likelihood surface. Multiplying that surface by the preceding posterior and normalising produces the next posterior—the same sequential rule as before, now across a two-dimensional grid.
Code detail: Sequentially updating the 41 × 41 Gaussian grid
The cache starts with one prior surface. For each required height, the browser computes all 1,681 likelihoods, multiplies them pointwise by the last posterior, normalises, and appends both surfaces.
(defn ensure-gaussian-step! [target-posterior-count]
(loop []
(let [{:keys [posteriors likelihoods]} @gaussian-cache
observed-count (dec (count posteriors))]
(when (< observed-count target-posterior-count)
(let [likelihood (height-likelihood
(nth adult-heights observed-count))
posterior (normalize-mean-one
(mapv * (peek posteriors) likelihood))]
(reset! gaussian-cache
{:posteriors (conj posteriors posterior)
:likelihoods (conj likelihoods likelihood)})
(recur))))))Caching avoids recomputing earlier steps. Previous and Next only change which already reproducible update is displayed.
View the complete Gaussian-grid implementation and adult-height data
Predict before revealing: what will repeated heights do to the grid?
Before pressing Next, predict whether plausible parameter pairs will spread across the map or concentrate. Which direction should a relatively short height pull the mean?
Reveal by stepping: the likelihood for one height is broad, but repeated multiplication concentrates posterior support where one mean–spread pair explains the complete dataset reasonably well.
How to read the height simulator
- Horizontal position is candidate mean μ; vertical position is candidate standard deviation σ.
- Within each panel, stronger opacity means greater relative support. The scale is logarithmic.
- Compare location and concentration, not the absolute shade between different panels.
- Read left to right: posterior before this height, this height's likelihood, posterior after the update.
Loading the sequential Gaussian updating simulator…
Build: cross candidate means and standard deviations into a two-parameter grid. Check: inspect each likelihood and the before/after surfaces. Decide: retain the complete posterior surface, not only its highest cell.
4. Make the lesson reproducible
An executable article joins an explanation to calculations that can be run again. Source code is the human-readable set of instructions stored in the .clj and .cljs files. A runtime is the software environment that executes those instructions. Browser state is the current in-memory data behind controls, such as accumulated tosses or the selected step.
A render turns source into a reader-facing artifact. Here Clay evaluates the Clojure article using Kindly’s display conventions and writes QMD (Quarto Markdown); Quarto turns QMD into HTML; Scittle executes the ClojureScript in the browser; Reagent updates semantic HTML and accessible SVG as browser state changes.
The source-to-browser path. Clay, Scittle, and Quarto behavior follows their official documentation; links appear in Sources.
Accessibility means people can perceive and operate the article through different senses and input methods. Each SVG has a programmatic title and description; every control has a visible label and keyboard behavior; colour is paired with position, text, line style, or opacity; layouts reflow on narrow screens.
A regression check asks whether an established property still holds after a change. The assertions below protect the 201-point grid, combinatorial count, normalisation, decision, and positive density. The browser checks add interaction, console, layout, focus, label, and theme verification. A random seed makes simulations replayable, so a changed result can be distinguished from ordinary random variation.
Code detail: Mounting all three browser components
A mount point is an empty, uniquely identified HTML element reserved for an interactive component. On page load, the browser finds each preserved ID and asks Reagent to render the matching component there.
(defn ^:export mount []
(when-let [root (js/document.getElementById
"globe-update-simulator")]
(rdom/render [globe-update-simulator] root))
(when-let [root (js/document.getElementById
"posterior-sampling-simulator")]
(rdom/render [posterior-sampling-simulator] root))
(when-let [root (js/document.getElementById
"gaussian-height-simulator")]
(rdom/render [gaussian-height-simulator] root)))The conditional lookup lets the same source load safely even if one mount is absent. The three IDs remain unchanged from the published article.
The next article reuses the same structure: candidate knowing rates receive priors; quiz responses supply likelihoods; posteriors predict untested items; seeded draws describe uncertainty; and an explicit stopping decision depends on the intended measurement task.
Sources
- Jamie Pratt and the JointProb group, original interactive simulations and ClojureScript source.
- Richard McElreath, Statistical Rethinking, examples from sections 2.2, 3.2, and 4.3. The prompts here are paraphrases.
- Clay documentation on executable Clojure documents and Quarto output.
- Scittle documentation on running ClojureScript in a browser.
- Quarto HTML documentation on rendering HTML output.
- W3C Web Accessibility Initiative, SVG accessibility guidance and accessible names and descriptions.
Regression checkAll executable assertions passed.