Singular Learning Theory: AI learns like ice melts

Intuitively, neural networks should not work but they clearly do. Why is that? And what does it have to do with ice?