This gloss of a key passage from “Computer Systems: A Programmer’s Approach” is inspired by Poul Anderson’s “Uncleftish Beholding” . A reckoner bundle is made of hardware and bundlish software that work together to run wending riddlelists. Enlightenment within the reckoner is forstood as clusters of twines (short for two rimes ) that are understood in unlike ways, depending on the background.…
Relational databases like postgreSQL and operating system kernels like Linux are foundational building blocks for computing systems. Every “application” is in essence an “application” of those building blocks to some purpose: Twitter applies databases to write and read tweets; it applies an operating system to operate the hardware required to do so. Large language models – and large generative…
Here’s a Rust Riddlelist that bids the world hello: fn main () { println! ( "Hello, World!" ); } Let us walk through the words of this riddlelist step by step and come to ken how it drives the reckoner to weave a greeting. Do not — as some ungracious pastors do — Show me the steep and thorny way to heaven, Whilst, like a puffed and reckless libertine, Himself the primrose path of dalliance treads,…
tl;dr GLM-130B is a GPT-3-scale and quality language model that can run on a single 8xA100 node without too much pain. Kudos to Tang Jie and the Tsinghua KEG team for open-sourcing a big, powerful model and the tricks it takes to make it run on reasonable hardware. Results are roughly what you might expect after reading the paper : similar to the original GPT-3 175B, worse than the InstructGPTs.…
As part of the new and improved 2022 edition of Full Stack Deep Learning , I delivered the lecture on ethics – in technology broadly, in ML more narrowly, and in the emerging field of artificial intelligence. You can find the lecture notes on the course website here .
As part of the new and improved 2022 edition of Full Stack Deep Learning , I delivered the lecture on troubleshooting and testing. You can find the lecture notes on the course website here . On this blog, I wanted to write a bit about why this topic – which most people find a bit dull – is so important to me. My intellectual background is not in software engineering. It’s in the sciences of…
As part of the new and improved 2022 edition of Full Stack Deep Learning , I revamped the demo project that is built through the “labs” portion of the course. You can watch a walkthrough of the lab overview on YouTube: And you can read more about the labs on the course website here .
In this book, you’ll learn a holistic approach to designing ML systems that are reliable, scalable, maintainable, and adaptive to changing environments and business requirements. As part of the ML Collective independent research organization, I led a reading group of the soon-to-be-released O’Reilly book Designing ML Systems , by Chip Huyen . You can catch the recordings on YouTube here .
Different programming languages push us to write code differently. Python pushed me to write clean code. Rust pushed me to write performant code. Haskell pushed me to solve problems differently. Three Borrowed Refs, Two Left Folds, & A Python In A Binary Tree Every year, Advent of Code puts out 25 coding puzzles in December, one a day until Christmas. I first did AoC in 2015 , when I was just…
But now that Weights & Biases can render PyTorch traces using the Chrome Trace Viewer, I’ve decided to peel away the abstraction and find out just what’s been happening every time I call .forward and .backward. These traces indicate what work was being done and when in every process, thread, and stream on the CPU and GPU. By the time we’re done here, we’ll have a clear sense for why most of the…