It’s Finally Done After almost six years, createlang.rs is complete. The book on creating your own programming language with Rust is finished. Not “mostly done” or “just needs polish.” Actually finished. I can barely believe I’m writing this. How It Started Back in 2020, I tweeted about the project: Can’t see the embed? View the tweet on X → At the time, I was excited. I had this vision of a…
The Paper Alright, so I finally finished writing this up. Here’s the paper if you want to dive straight in (I like coffee!): Your browser doesn't support embedded PDFs. Download the PDF here Download Paper (PDF) Acknowledgments Huge thanks to Chris Lattner and Abdul Dakkak for taking the time to review drafts of this paper. Your feedback was incredibly helpful and pushed me to clarify and…
Mojo GPU Puzzles Edition 1 is out. It’s a set of 34 puzzles that teach GPU programming, and the whole premise is that you write code from puzzle one instead of reading theory for three chapters first. Why Mojo? A few reasons. The syntax is close to Python but the performance is closer to C, and the type system catches a lot of the mistakes you’d normally only find at runtime. The bigger draw for…
I wrote a post on the Modular blog about two optimizations that just landed in MAX Serve: paged attention and prefix caching. Both attack the same problem from different angles, which is that the KV cache in multi-head attention eats memory and recomputes work it doesn’t have to. In the article I explain what each one does, how to turn them on, and what the speedups look like in practice. Read it…
I wrote a post on the Modular blog about building AI agents with MAX Serve and OpenAI-style function calling. The idea behind an agent is simple enough: give the model a set of tools it can call, and let it decide when to call them. Getting that loop to behave in production is the harder part. The article builds a first agent from scratch and then covers what changes once you want to actually ship…
I wrote a hands-on post on the Modular blog about Mojo 24.6. The big changes here are around argument conventions and reference tracking: there’s a new mut keyword for mutable arguments and an origins system that the compiler uses to track references. There are also some new collection types. I worked through each one with code you can actually run. Read it here: Hands-on with Mojo 24.6 .
I wrote a tutorial on the Modular blog that builds a chat interface on top of Llama 3 and MAX Serve, end to end. It covers serving the model, keeping a running conversation going, and the part people usually get wrong: managing context and history as the chat grows. It finishes with how to deploy the thing. Read it here: Build a Continuous Chat Interface with Llama 3 and MAX Serve .
I wrote a post on the Modular blog about two optimizations that just landed in MAX Serve: paged attention and prefix caching. Both attack the same problem from different angles, which is that the KV cache in multi-head attention eats memory and recomputes work it doesn’t have to. In the article I explain what each one does, how to turn them on, and what the speedups look like in practice. Read it…
I wrote a hands-on post on the Modular blog about Mojo 24.5. Rather than just listing the changelog, I tried to show what the new language and syntax features actually look like in real code, what they do for performance, and how to update code written against earlier versions. Read it here: Hands-on with Mojo 24.5 .
I wrote the MAX 24.4 release post on the Modular blog. The headline for this one is that MAX now runs on macOS, so you can do fast local Llama 3 inference right on your laptop. It also adds native quantization and GGUF support, which means a lot more models work out of the box. The article goes through each of these with benchmarks. Read it here: What’s New in MAX 24.4? .
I wrote a post on the Modular blog about ownership, mostly in the context of Mojo. Ownership trips a lot of people up, and I think that’s because we tend to learn the syntax before we have a mental model for what it’s actually doing. So that’s what the article is: a mental model for ownership, why it lets you write safer code, and how Mojo’s take on it compares to what you might know from Rust or…
I wrote a post over on the Modular blog about building semantic search with MAX Engine. Unlike keyword matching, semantic search compares the meaning of a query against your documents, so it can find relevant results even when the exact words don’t line up. The article walks through generating embeddings with MAX Engine and wiring them into a search pipeline, with working code and some notes on…
Prelude In this post, we’re going to deep dive into one of my favourite tools that are revolutionizing how Python code is run in cloud and it’s especially aimed at the computing stack for Machine Learning / Deep Learning applications, called Modal which has recently gone GA! There are almost no resources besides the official documents and examples. I have been using Modal for almost a year and…
Over the past year, perhaps the most cited paper across the software industry is Attention is All You Need that is at the heart of ChatGPT and GPT transformer models. The first thing you will notice in the paper is the Attention formula: Unfortunately, very few sources have delved into where this has come from i.e. the core of the attention mechanism, and most explanations provide little to no…
Prelude In the Rust world, the interaction between Python and Rust is very well-known through the amazing PyO3 ecosystem. There is a similar relation between Python and Javascript in particular Node.js that I’m going to describe in this post. All the code is available here . Most programming language interactions happen through C layer ABI i.e. FFI. However, interacting Rust with JavaScript is…
This post tries to shed some light on the rapidly changing LLM frameworks in particular, LangChain (LC) and Llama-index (LI). Library vs. Framework It’s tricky to draw a clear boundary between a package/library and a framework, but for the sake of discussion, let’s look at some well-known examples Packages: Numpy falls into this category. It provides functionality that can be adapted to various…
DLPack is the standard in-memory data format that facilitates zero-cost tensor transfer across major Deep Learning frameworks (PyTorch, TensorFlow and TVM) and the supported Python Array processing frameworks such as Numpy, CuPy. The dlpackrs provides a safe idiomatic Rust binding where Rust ndarray and tensor frameworks can use it to gain the same kind of benefits (if not done already) across the…
If you happen to write unsafe code in Rust where normal static checks are not available and want better UX for detecting memory issues along side using various sanitizers, checkout my new crate smartalloc which provides idiomatic Rust binding for the original C version here . Beside the reason in README, note that MIRI can’t be used now since it doesn’t support FFI function call. With Rust…
After almost a year from my last blog post, in this short post I’m very happy to announce that I’m writing a free online book where early chapters are available now. I’ve explained my motivations and goals in the introduction. The accompanying codes are also available on my GitHub . Feedbacks are welcome and happy learning. If you’ve found the book useful, please consider donating to any of the…
This time we dive into std::pin which has a dense documentation. Types that pin data to its location in memory . It is sometimes useful to have objects that are guaranteed to not move , in the sense that their placement in memory does not change , and can thus be relied upon. A prime example of such a scenario would be building self-referential structs , since moving an object with pointers to…