Mini-Retirement: Or, How I Learned to Stop Grinding and Took Two Years Off
Sunrise over Mt. Bromo, Java, Indonesia
Neural Pensieve
A personal space for sharing my thoughts, experiments, and discoveries in AI.
Latest posts
The Art of Safe Policy Updates: From REINFORCE to TRPO and PPO
Policy gradient methods are like climbing a treacherous terrain to find the highest peak. Better use a harness like TRPO or PPO!
Teaching AI to Play Hokm: A Multi-Agent Reinforcement Learning Challenge
Hokm is a 4-player Persian card game similar to Bridge, and the robot is learning it by playing against younger versions of itself!
Teaching (tiny) LLMs to Play Text-Based Games Using RL (on a $300 GPU)
TL;DR: I trained small LLMs like GPT-2 to play text-based games using reinforcement learning (PPO) on my budget GPU. Key innovations: token-level PPO for richer signals and trie-based masking to ensure only valid actions. It worked well on with various levels of difficulty, with techniques applicable to real-world structured outputs. Code on GitHub, results show generalization, and I share lessons…
The Beautiful Intuition Behind Diffusion Models
The first time I heard about how diffusion models work, I was dumbfounded! How can you start from random noise, denoise it over and over, and get a clean generated image? In other words, how can no information result in a lot of information? There are many papers and blog posts explaining the math behind this, and honestly, it may even seem intimidating at first, but when you understand the…
How to Tame Your Deep RL
Running reinforcement learning (RL) experiments—especially when implementing models yourself—is notoriously more challenging than typical supervised learning (SL) experiments (as attested by none other than Andrej Karpathy here ). SL models are generally well-behaved: they converge predictably, respond well to cross-validation, and scale efficiently on GPUs. Deep RL models, however, are entirely…
How many words do you know?
A few years back, I started wondering: what’s the minimum number of questions you’d need to ask someone to figure out the size of their vocabulary? Of course, asking random questions is not exactly optimal, and clearly the best solution would involve some kind of exploration-exploitation trade-off, but I never got around to actually work on it. Now that I’ve got some free time, I figured I’d give…
Perils and Promises of AI
With the rapid rise of Generative AI and its growing influence, humanity is entering an era of uncertainty, particularly concerning AI’s negative impact on society. The unprecedented capabilities of AI are both exciting and alarming, as they hold the potential to radically transform industries, economies, and human interactions—for better or worse. It is not hard to imagine a future where AI…