Equivariance in Neural Networks: A Free Lunch That Isn't

Some inductive biases encode beliefs about the data: bets that certain hand-crafted features matter, that certain regularities hold. These can be wrong. But some encode theorems about the target function. These cannot be wrong. Rotational equivariance in molecular simulation is one of them. This post is about why encoding a provably correct theorem into a neural network is harder than it sounds.…