2024-8-25: Scaling curves for All of the Things
Good news: we got a bunch of important findings this week.
Davis Summarizes Papers
I go through all the machine learning arXiv submissions each week and summarize 10 to 20 of my favorites. Free forever and read by thousands of ML researchers and practitioners.
Subscribe:.rss.atom.json.md.m3u.pls
Dormant Last read · last published · next check
Read 10 days ago and current, but nothing has been published for 2 years.
Latest episodes
2024-8-4 arXiv roundup: LLama 3.1, training a 100T biological neural net
In case you’re wondering what I’ve been up to instead of posting for the past couple months, I was kicking off a training run for a 100T parameter biological neural network:
2024-4-28 arXiv roundup: data and scaling, backlog highlights part 3
Besides getting to cover unusually interesting work, the upside of having a big backlog is that you can group your coverage thematically.
Research Archetypes: Scientists, Mathematicians, Inventors
What "research" entails day-to-day varies by field, subfield, problem, and individual researcher.
2024-4-14 arXiv roundup: backlog highlights part 2
Bunch of interesting stuff this week. Before we jump in, one quick clarification from last week: I mentioned how it was an interesting marketing lesson from DBRX development how we spent a bunch of time adding to MegaBlocks but then people ended up associating it with Mistral because they released an MoE first. A couple people said this part was “super spicy” (maybe because of the phrasing of the…
2024-4-7 arXiv roundup: DBRX, Backlog highlights part 1
It’s good to be back.
2023-11-26 arXiv roundup: Big potential wins, 1 bit per parameter, Simplifying transformers
Stella Nera: Achieving 161 TOp/s/W with Multiplier-free DNN Acceleration based on Approximate Matrix Multiplication
2023-11-19 arXiv roundup: Inverse-free inverse Hessians, Faster LLMs, Closed-form diffusion
A fundamental result in queueing theory is that, if items enter the queue faster than they’re processed, the length of the queue tends to infinity.
2023-10-16 arXiv roundup: Cornucopia of easy (claimed) wins for LLMs
Also, I was on the AI Stories podcast!
2023-9 arXiv roundup: A bunch of good ML systems and Empirical science papers
Got behind the curve again and ended up taking me more than a week to catch up.