Deep Learning and Robotic Rambles. AI blog by Josh Levy-Kramer.
An interactive visualisation of Jensen–Shannon divergence - the symmetric, always-finite cousin of KL. Shape two distributions and watch JSD, its ceiling of one bit, and the per-point contribution respond in real time.
An interactive visualisation of Kullback–Leibler divergence. Shape two distributions and watch forward vs reverse KL, the pointwise integrand, and the effects of asymmetry, support mismatch and discretisation.
Obtaining the gradient of the matrix inverse
Obtaining the gradient of the matrix inverse
Obtaining the gradient of the Cross-entropy loss (softmax and negative log-likelihood loss function