My personal blog
Over the past few weeks I’ve spent a lot of time reviving some of my older machines, installing many different distros and tweaking them and I wanted to share my 1-line reviews in case they help someone else looking to take the plunge into Linux for desktop. TL;DR what does Mark use Pop!_OS : for my main desktop which I use both for local development of PyTorch and gaming Omarchy : on my…
If you’re excited about optimizing code that runs equally well on a single or thousands of GPUs and if you have the ability to submit a single substantial PR to a major OSS library, we want you on the PyTorch team - especially if you’re early in your career. Why PyTorch? Both LLaMA and ChatGPT pretraining rely on our infrastructure, so do thousands of other AI workloads and companies.…
Why won’t Llama 13B fit on my 4090 I recently gave a talk that a bunch of people liked, maybe I’ll write a proper blog post at some point but in the meantime, if you care to learn about the different kinds of memory overhead in Neural Networks and how to ballpark them I’ll bet you’ll enjoy this talk
A batch’s journey: torchdata internals The core code is in torchdata . About 3237 lines dedicated to datapipes and about 376 lines deddicated towards dataloaderv2 but the repo is moving fast and this document may be out of date in a few weeks with regards to specifics but not core ideas. torchdata is fundamentally well about loading data into PyTorch with the familiar iterator pattern we all…
Reinforcement Learning is a Game Design Problem 2 pager describing how I’d like the field to move towards environments that are differentiable, multi-agent, compositional, multi-modal , continuous and self supervised RL is Game Design
Applications of Reversible Computing I first came across reversible computing back when I was helping organize a quantum computing seminar back in grad school. At the time I didn’t think much of it, I knew that for some quantum mechanical reason gates had to be reversible but I didn’t really appreciate why that was such a powerful idea at the time until I came accross reversible…
A personal review of Zach-like the book This was the first book I ever bought on Twitter. The book had a kickstarter that I missed but thankfully Zachtronics had been spamming my feed saying there are more books available and I honestly feel really fortunate to have a book like this in my library. If you’re reading this, I’d imagine you have played or at least heard of at least one…
Why developers should also be product managers This post is summarizing some observations I’ve made working at Microsoft as a Program Manager, Applied Scientist and Developer. My intention is to go beyond simple narratives like “developers don’t know how to focus on the big picture” and “product managers are clueless middle managers” since it’s easy to…
Work in Progress Representation Theory for Robotics This goal of this post is to teach you how to efficiently represent the state of a robot with as little memory as possible. The primary application of this is that we can then train Reinforcement Learning (RL) algorithms orders of magnitude faster. One of the main reasons isn’t the de facto standard in industrial applications is that the…
Controlling a 2D robot arm with deep reinforcement learning and Floydhub My new post is on the Floydhub blog . Had a great time collaborating with the team. Check it out if you’re bored of predicting rent prices when you’re trying to learn about machine learning.