Explanining Log-Sum-Exp

Motivationn & Definition In this post, we explain the concept of Log-Sum-Exp . Suppose we have three extremely small numbers: $a_1 = e^{-200}, a_2 = e^{-201}, a_3 = e^{-202}$ . We are interested in the log of their sum. There are two issues to solve here: The sum will still be very small. To make the result more readible, we want to take the logarithm of it. Dealing with extremely small numbers…