Some scientists believe the performance of LLMs have plateaued. With this in mind, we can ask a critical question
Is it time to explore alternative architectures?
Today’s LLMs are black-box, which means nobody knows how they work internally. This may seem fine if we need an oracle that just answers questions, and from a performance perspective, there is no theoretical reason that a black-box will under-perform a white-box model. For example, Dr. Yann LeCun in a podcast incorrectly assumed an LLM cannot learn physical phenomena without interacting with the physical world.
It is tempting to view this model as the path to AGI. However, a black-box model brings out a lot of questions.
Research/Engineering Perspective
Theorem proving is one of the most rigorous tests of reasoning, and an LLM cannot just output text which looks correct. For this reason, I consider theorem proving to be a standard benchmark for reasoning capabilities in the LLM. A fair assessment of reasoning would entail training the LLM on one domain (e.g. Algebra) and testing it on another domain (e.g. Analysis). From my experience with LLM-based MATP methods, their performance is abysmal in this field.
State-of-the-art LLM-MATP works use test time RL which is computationally intensive. Dr. Terence Tao has written a thread about this, so I’ll not repeat it here.
One of the most irritating thing about working with LLM is when they fail, we don’t know why. This is the primary drawback of a black-box model. From this, there are 3 reasons why we want a white-box model:
- Make failures easy to diagnose
- Dispel cargo-cult mentality associated with today’s LLMs. e.g. this kind of paper, and this kind of paper, and the general aura of “if we can feed more data into LLMs then we can solve every problem.”
- Potentially shrink the size of models.
The requisite computation demand for training and inference on LLMs poses a high barrier to entry for smaller labs and independent researchers. This leads to a question
Does a model need to know everything on the internet or read millions of books to do useful work?
Societal/Ethical Perspective
IMPORTANT
I am not a rationalist or effective altruist. I consider such movements to be naïve and sometimes cultish.
This is what I believe to be the most urgent reason to develop white-box models.
Many “AI Dangers” thought experiments are based on the assumption that the AI is a black-box. This assumption is natural, since we cannot see inside of the minds of other people we are interacting with. However, when we interact with other people, our level of mutual trust is based on shared experiences and memories as a species, where as with machines we do not have this shared history. For this reason, a white-box model can
- Increase people’s trust in artificial intelligence
- Solve the intention problem, where the instructions to the AI is not carried out as intended.
- Ensure safety
Moreover, the massive computation budget leads to huge resource demands from data centres at the hands of a few companies. If we do not want to see a future where only OpenAI, ByteDance, etc. have an oligopoly on AI, we need to develop a new generation of models.