Jon Eskin's programming notes and blog posts
Posted on May 16, 2025 by Jon Eskin Introduction In a recent project , I compared output from HuggingFace’s Candle with PyTorch equivalents to make sure that embeddings and calculations involving them are behaving as expected. This led to comparing Python snippets that look like this: def normalize_l2(embeddings): return F.normalize(embeddings, p = 2 , dim = 1 ) to Rust that looks like this: pub…
Posted on September 27, 2024 by Jon Eskin Introduction Reproducing GPT-2 from scratch is a great way to build an understanding of deep learning and large language models. A number of learning resources about GPT-2 architecture are available, including Andrej Karpathy’s free Zero to Hero course and Sebastian Raschka’s book Build a Large Language Model from Scratch . While these resources provide…
Posted on August 12, 2024 by Jon Eskin Introduction In a previous post , I explored using LLMs to perform text classification tasks. The idea was that they could enable automation of more complex tasks that are otherwise not automatable. Many people just use a model like ChatGPT for this, but after learning a bit about how they work, I was getting the sense that other approaches might be more…
Posted on July 6, 2024 by Jon Eskin Update 2025-05-18: Looking back, this post is a little embarassing, but I’ll leave it up in case any other learners stumble across it and find it helpful. In the video The spelled-out intro to language modeling: building makemore , Andrej Karpathy briefly mentioned a bug that can occur when broadcasting tensors during mathematical operations. In the lines below,…
Posted on June 30, 2024 by Jon Eskin I’ve encountered many tasks that could almost be automated, except that they have some kind of ambigious step that requires manual attention. For example, it’s extremely common to find instances where freeform text needs to be flagged if it contains sensitive or important information. For such tasks, it’s easy to write software that performs subsequent steps…
Posted on June 20, 2024 by Jon Eskin ollama is a software project which makes it easy to run LLMs on your local machine. Running ollama run llama3 downloaded a 4-bit quantized model that could run on my Macbook M2, and then dropped me into a CLI where I could enter prompts and responses stream into my terminal. Your browser does not support the video tag. In the above clip, I’m running Meta’s…
Posted on April 16, 2024 by Jon Eskin A couple of years ago, I worked on an exploratory protoype of web application to get my hands dirty with Clojure and Datomic. I put together a prototype web application with a frontend written using Tailwind CSS and Alpine JS and a backend written using Pedestal and Datomic. I chose to hand-roll everything including authentication and role based access control…
Posted on November 26, 2023 by Jon Eskin Meta Quest headsets let you write 3D programs in C or C++, using OpenGL ES or Vulkan as a rendering backend, and deploy them using the XR Mobile SDK . I bought a Quest 3 at launch, and I’ve benefited a lot from studying the SDK and experimenting with the sample programs: It gave me a thorough understanding of the OpenXR specification, from its design intent…
Posted on January 14, 2023 by Jon Eskin Introduction In a recent whitepaper , researchers described a technique to take existing pre-trained text-to-image models and embed new subjects, adding the capability to synthesize photorealistic images of the subject contextualized in the model’s output. A series of implementations were quickly built, finding their way to the Stable Diffusion web UI…
Posted on January 3, 2023 by Jon Eskin Introduction lchat is a small program that is designed to interface with other programs such as ii that present FIFO interfaces. This blog post will briefly show what that means and how lchat works in practice. ii A typical irc client works by negotiating a connection with an IRC server before presenting the user with some kind of interface to interact with…