Deep dives on modern AI and computers
Model collapse is the phenomenon in which text or images output by AI are uploaded to the internet, those outputs become mixed into AI training data, the AI trained on that data then produces further outputs that are uploaded, and this cycle repeats until AI performance collapses. The most famous study is probably AI models [ ]
What I like about this result is that it combines a strong wait, you can do that? feeling with real research usefulness. I recently used it in our paper, and since ChatGPT could not answer it when I asked, I decided to write this post. The original source is Dynamic Generation of Discrete Random Variate. [ ]
If there are a million people, then there will be someone who flips heads 20 times in a row. Even when there is no difference in how many chances people are given and no difference in how lucky they are, and the playing field is perfectly equal, some people will still end up looking lucky. [ ]
If you are unsure what to do in the age of AI, it is good to look to the past.When you think about what kind of life you have lived up to now, a path opens up. AI levels people s abilities.With coding agents, the gap between the very best programmer and an ordinary programmer becomes [ ]
AI performance is improving day by day, and its evolution may appear unstoppable. However, in reality, that is not what will happen. In this article, I explain why. The key terms in this article are epistemic uncertainty and aleatoric uncertainty. Epistemic uncertainty is uncertainty that can be resolved through observation, whereas aleatoric uncertainty arises from [ ]
Cognitive semantics is a linguistic approach that understands the meaning of words in relation to human cognition and ways of perceiving things. In this article, I would like to talk about the cognition of LLMs from the perspective of cognitive semantics. In the background of cognitive semantics are fields such as Gestalt psychology and cognitive [ ]
I think a lot of people feel a real sense of crisis about AI taking their jobs. I feel it too. In fact, I am in the almost absurd, joke-like situation where I was supposed to be working on building AI, and now even that very job looks as though it may itself be taken [ ]
We can flatten the parameters of a deep neural network into a single vector. For a large model, this vector can easily have tens of billions of dimensions. At first glance it may look like a meaningless list of numbers, but it has become clear that this vector has deep meaning as a vector. For [ ]
Over the past few years, LLMs have developed rapidly, significantly expanding our cognitive capabilities. Many people probably now turn to AI first when they encounter a problem. It may be that LLMs already know more than my brain does, and perhaps possess higher reasoning abilities than my brain. However, in this article, I will discuss [ ]
In this article, I will argue that transformers are RNNs. I will not merely point out formal containment; I will also explain how similar transformers and RNNs are, and what practical implications follow from that viewpoint. Transformers are RNNs First, I will show that transformers are RNNs. To put it simply, a transformer is an [ ]