LLMs are complicated now

Back in 2022 and 2023 there were two big branches of machine learning happening at Meta. The LLM work that led to Llama was a clean, smooth stack of repeated Transformer modules; the recommendation systems graphs were, by contrast, terrifying. Luckily, the industry has remedied that state of affairs by making LLMs a lot more [ ]