On the relationship between sigmoid, softmax and tanh

If you have even a passing interest in deep learning, chances are you’ve encountered at least one of these functions: sigmoid, softmax, and tanh. Over the past few decades, they have been integral to numerous research efforts and will likely continue to be so for many years to come. What’s truly remarkable is how few people know about the relationship between these functions, and even top…