Recent content on Saheb's AI Journey
It’s one thing to be aware of Sutton’s bitter lesson , quite another to internalise it; and even harder to resist encoding human bias, that we might think of as human knowledge into our agents. It’s not at all hard to notice the beauty of it; it’s omnipresent and most of us know that the role of the teacher is not to give the answer, but to teach the student how to find the…
An AI agent trained on a smaller board should be able to play well on a big board. Train on 6x6, or 10x10 and test it on 100x100. If we can do it, so should an AI agent. Simple concepts to learn: move closer to the apple do not collide with the wall do not collide with yourself (body) I didn’t expect to learn breadth and depth of RL from this toy problem, but I was pleasantly surprised. I…
Every architectural layer is a compression operator. Before using it, ask what information it intentionally throws away, what assumptions justify throwing it away, and whether those assumptions still hold under the distribution shifts you care about. This principle applies far beyond pooling—to convolutions, transformers, graph neural networks, positional encodings, normalization layers, residual…
Source: Kirk et al. 2023 ZSG survey (arXiv 2111.09794) + working sessions, July 2026. 1. The problem in one sentence Train a policy on a finite sample of contexts drawn from p(c); freeze it ; measure expected return on contexts never trained on. Zero-shot = no adaptation at test time (no gradient updates, no practice episodes — nothing to do with “one attempt”). Generalization gap =…
In the second part we talked about transformer heads and features. In this part, we will dive into credit assignment and reward hacking problems that I faced. Giving +1 reward when you win a game and -1 when you lose is clean and eventually you want to reach that state where it suffices but before you get there, you need more signals, i.e. dense rewards to teach the agent the behavior you want.
In the first part, we discussed self play and the equilibrium our agent gets stuck in. In this part, we will talk about features that model takes as input, and how model uses them to learn strategies. Theme and tone will be the same, more on what not to do. In Orbit Wars, features fell into 4 categories, planets, fleets, global, and pairwise features. Planet features will tell the model everything…
I had a lot of fun building an AI agent for Orbit Wars. But I failed, according to my own standards, and naive high expectations. I wanted to be in top 10. I didn’t even end up in top 1000. The final scoreboard: a rating of 995, ranked 1060 out of 4729. Top 25%, but nowhere near the top 10. Getting there took over 90 training runs across a month, and about $700 worth of GPU compute across GCP,…
Building reinforcement learning environments is already a complex exercise in managing state, rewards, and training loops. You expect the agent you are training to act unpredictably. You don’t expect the AI agent writing your code to completely flatline. While working on some recent RL projects, my AI code editor—Google Antigravity—just gave up. Every time I hit send, it would emit a single…
Reinforcement learning is interesting because it is intuitive and it aligns with how we learn naturally. You touch hot water, you feel the burn, you don’t touch it again. You learn by trial and error; you learn from experience. By interacting with the world, you get feedback in the form of reward or punishment, and that defines your behaviour. You notice there is a knob next to the water…