Towards artificial sentience.
A major trend of this year has been vibe coding using LLMs to create software from the ground up, without a human ever interacting with the source code directly. The current consensus is that vibe coding isn t quite ready yet for developing critical production software. But while those issues are being worked out, we [ ]
Assume we want to use a vision-language model (VLM) to look at a given image and determine certain properties about it. Let s say that, for the sake of the argument, we would like it to determine whether the image contains anything funny or not. As we all know, jokes are the most funny when you [ ]
This article is part of the series Permanence might be all you need . The series documents a side project of mine in which I explore invariant learning. Also in this series: From invariant representations In machine learning, we often represent the state of a system as vectors. Artificial neural networks take the vector representation of [ ]
This article is part of the series Permanence Might Be All You Need . The series documents a side project of mine in which I explore invariant learning. Also in this series: We need priors to learn If you re already familiar with inductive priors and the No Free Lunch Theorem, feel free to skip this section. [ ]
The notion of object permanence, the understanding that objects continue to exist even when they cannot be seen , is considered an important developmental step in young children, and often used as an indicator of advanced mental capabilities in animals. But could permanence play a much more fundamental role in how intelligence forms? Could it in [ ]
Imagine an artificial organism living in a complex environment. The organism is not just a passive observer. It can decide which parts of its environment it wants to explore next. It might even be able to actively influence the world around it by taking various actions. We want the organism to learn from its experiences [ ]
In my previous post about ThoughtNet, an attention-based neural architecture for variable-compute inference, I highlighted two limitations that I encountered with it: While trying to solve the second problem, I stumbled across a surprising way to stabilize training convergence as well. Was ThoughtNet Cheating? While attempting to understand why my ThoughtNet models weren t generalizing much [ ]
The majority of today s artificial neural network (ANN) architectures perform a constant amount of computation at inference time regardless of their inputs. This includes all recent GPT-style LLMs1 and other transformer-based architectures. Whether you ask an LLM to complete the series 1, 2, 3, , or you ask it to solve a complicated logic riddle, [ ]