The Bitter Lesson is an excellent essay which is overwhelmingly misunderstood. The point of the bitter lesson is that, over time, methods which scale with compute will outperform methods that do not. It is not: The idea that we should never incorporate human knowledge The idea that deep learning and scale are all we need (Rich is actually relatively skeptical of deep learning) The entire point of…