Note/Moore's Law — AI Scaling Law

»The [AI] scaling law indicates that the performance of Large Language Models (LLMs) can continually improve as the model size increases« 1; it is reminiscent of »Moore&rsquo;s Law« 2 in the context of computer architecture. »Moore’s Law« (if we are honest, one ought to call it »Moore’s Observation« as it isn’t a physical law but rather an observation, reformulated as a prediction, that the number…