An estimation procedure that is often claimed to be part of Bayesian statistics is the maximum a posteriori (MAP) estimate of an unknown quantity, that equals the mode of the posterior density with respect to some reference measure, typically the Lebesgue measure. The MAP can be used to obtain a point estimate of an unobserved quantity on the basis of empirical data. It is closely related to the method of maximum likelihood (ML) estimation, but employs an augmented optimization objective which incorporates a prior density over the quantity one wants to estimate. MAP estimation is therefore a regularization of maximum likelihood estimation, so is not a well-defined statistic of the Bayesian posterior distribution.
Assume that we want to estimate an unobserved population parameter
\theta
x
f
x
f(x\mid\theta)
x
\theta
\theta\mapstof(x\mid\theta)
is known as the likelihood function and the estimate:
\hat{\theta}MLE(x)=\underset{\theta}{\operatorname{argmax}} f(x\mid\theta)
is the maximum likelihood estimate of
\theta
g
\theta
\theta
\theta
\theta\mapstof(\theta\midx)=
| f(x\mid\theta)g(\theta) | |
| \displaystyle\int\Thetaf(x\mid\vartheta)g(\vartheta)d\vartheta |
where
g
\theta
\Theta
g
The method of maximum a posteriori estimation then estimates
\theta
\begin{align} \hat{\theta}MAP(x)& =\underset{\theta}{\operatorname{argmax}} f(\theta\midx)\\ &=\underset{\theta}{\operatorname{argmax}}
| f(x\mid\theta)g(\theta) | |
| \displaystyle\int\Thetaf(x\mid\vartheta)g(\vartheta)d\vartheta |
\\ &=\underset{\theta}{\operatorname{argmax}} f(x\mid\theta)g(\theta). \end{align}
The denominator of the posterior density (the marginal likelihood of the model) is always positive and does not depend on
\theta
\theta
g
g
When the loss function is of the form
L(\theta,a)=\begin{cases} 0,&if|a-\theta|<c,\\ 1,&otherwise,\\ \end{cases}
as
c
\theta
\theta
MAP estimates can be computed in several ways:
While only mild conditions are required for MAP estimation to be a limiting case of Bayes estimation (under the 0–1 loss function), it is not representative of Bayesian methods in general. This is because MAP estimates are point estimates, and depend on the arbitrary choice of reference measure, whereas Bayesian methods are characterized by the use of distributions to summarize data and draw inferences: thus, Bayesian methods tend to report the posterior mean or median instead, together with credible intervals. This is both because these estimators are optimal under squared-error and linear-error loss respectively—which are more representative of typical loss functions—and for a continuous posterior distribution there is no loss function which suggests the MAP is the optimal point estimator. In addition, the posterior density may often not have a simple analytic form: in this case, the distribution can be simulated using Markov chain Monte Carlo techniques, while optimization to find the mode(s) of the density may be difficult or impossible.
In many types of models, such as mixture models, the posterior may be multi-modal. In such a case, the usual recommendation is that one should choose the highest mode: this is not always feasible (global optimization is a difficult problem), nor in some cases even possible (such as when identifiability issues arise). Furthermore, the highest mode may be uncharacteristic of the majority of the posterior, especially in many dimensions.
Finally, unlike ML estimators, the MAP estimate is not invariant under reparameterization. Switching from one parameterization to another involves introducing a Jacobian that impacts on the location of the maximum.[2] In contrast, Bayesian posterior expectations are invariant under reparameterization.
As an example of the difference between Bayes estimators mentioned above (mean and median estimators) and using a MAP estimate, consider the case where there is a need to classify inputs
x
h1
h2
h3
x
h1
h1
x
x
Suppose that we are given a sequence
(x1,...,xn)
| 2 | |
| N(\mu,\sigma | |
| v |
)
\mu
N(\mu0,\sigma
| 2 | |
| m |
)
\mu
The function to be maximized is then given by[3]
g(\mu)f(x\mid\mu)=\pi(\mu)L(\mu)=
| 1 | |
| \sqrt{2\pi |
\sigmam}\exp\left(-
| 1 | \left( | |
| 2 |
| \mu-\mu0 | |
| \sigmam |
\right)2\right)
| n | |
| \prod | |
| j=1 |
| 1 | |
| \sqrt{2\pi |
\sigmav}\exp\left(-
| 1 | \left( | |
| 2 |
| xj-\mu | |
| \sigmav |
\right)2\right),
which is equivalent to minimizing the following function of
\mu
| n | ||
| \sum | \left( | |
| j=1 |
| xj-\mu | |
| \sigmav |
\right)2+\left(
| \mu-\mu0 | |
| \sigmam |
\right)2.
Thus, we see that the MAP estimator for μ is given by
\hat{\mu}MAP=
| \left( | ||||||||||||||
|
| 1 | |
| n |
| n | |
| \sum | |
| j=1 |
xj\right)+
| |||||||||||||||
|
| \mu | ||||||||||||||||||||||||||||||||||
|
.
which turns out to be a linear interpolation between the prior mean and the sample mean weighted by their respective covariances.
The case of
\sigmam\toinfty
\hat{\mu}MAP\to\hat{\mu}MLE.