The Gumbel-Max Trick

When we set out to learn a function or some property of it (like its maximum), we hope it is differentiable , because that means we have at our disposal a host of well-studied, and often fast, techniques. But sometimes we are not so lucky - and then there are broadly two options: (a) use a technique that doesn’t rely on differentiability, e.g., Bayesian Optimization, or (b) use an approximation…