Seemingly Small Changes with Big Impact in Modern AI

Having been living with GenAI and thinking about the transformer and MoE, which BTW was amazingly origined in this 1991 paper Adaptive Mixtures of Local Experts , the year another dinasour got his master’s degree in classical AI, I suddenly had the urge this Saturday morning to ask AI this question: “give me examples in the deep learning and genai field where one seemingly small change or adoption…