AI April: When a month feels like a decade

When I wrote my last newsletter on March 20, it felt like we had already reached Peak AI Hype, but in retrospect, it seems we were just getting started.

A random list of stuff that has happened since: we got open source models like Dolly 2.0 from Databricks, Elon Musk founded his own AI company, Sam Altman said the time of large language models is already over, Google merged DeepMind and Google Brain, Snapchat releases an AI chatbot but their app store rating tanked after it gets an overwhelming number of 1 star reviews, Palantir is showcasing AI supported warfare and says there is nothing to worry about, Dropbox is laying off 500 workers and pivoting to become an “AI company”,and then last week, Geoff Hinton, who invented backpropagation, came forward and said he has quit his job at Google because he needed to warn the world of the dangers of AI.

Probably the most impactful article was this leaked internal memo from Google that says that Google and OpenAI have not moat to protect them. The (leaked) foundational model LLaMa from Meta was the event that broke the dam. Early in March Meta had released the model for research purposes, but only a week later it was leaked and could be downloaded via bitorrent. This gave rise to projects like alpaca.cpp that allowed you to run it on your CPU, put it into the hand of almost everyone who had the patience to look for it. Since then, methods have been developed to fine-tune models with relatively modest resources. Stanford published a paper where they took an untrained LLM and about 150 examples of questions and answers, blew that up to a couple of ten thousand examples with an older GPT model and then fine-tuned the model to be almost on par with GPT. And all that for a couple of hours of GPU compute time.

I recently gave a talk for a client and my conclusion was also that things have changed, because suddenly we’re in a position where OpenAI doesn’t have a monopoly anymore but everyone can take one of the open source models, tune it, and adapt it to their own requirements for not more money than we have already been spending on training and running ML models in the past. We might not be fully there yet, but we’re getting close.

Interesting, one tool that has been indispensable for me to keep on track of what’s happening is a side project that I started back in January, fedistats. Originally I was interested in seeing what’s happening on the fediverse.

When Musk bought Twitter last fall there was this big exodus to Mastodon and there was constant threat to the availability of Twitter’s API, so I thought it would be time to take a look at Mastodon.

The project did a number of iterations, starting out with a searchable node directory, to a dashboard of the most active hashtags, till I finally built something to automatically compile an article feed for a topic like AI. It has been running since mid April and the list is already 200 articles long.

Recently I added a feature where you get all new articles once per day, so if you’re also suffering from massive FOMO on the latest AI news, you can try it by subscribing to fedistats’s daily AI summaries.

If not, look out for the next newsletter here :) I’m looking forward to what the next month will bring!