This is a short one, and an obvious one, but I feel like I should have absorbed it much earlier in my "I'm not an SRE I just need to get my problem solved" career. The other day I needed to run some command on a production box. The command does some stuff, then spits out a handful of CSVs including records of what it did. You know, for record keeping. So I logged onto the box, ran my_command ,…
From Python 3.14, python -m pdb -p pid lets you connect a pdb session to a running Python process. This post goes into a part of what makes this possible. The barrier to entry for writing general debugging tools for Python programs has always been quite low. Unlike many languages, you're rarely finding yourself working with weird internals. Instead, debugging tools can be built off of pretty…
Sometimes you're looking at a snippet of code, and want to change it. my_data my_data . some_attr When going from the first line to the second, I needed to put my cursor near the end, and then add the attribute access. Especially when working in a REPL where the previous line was just above me, this is pretty nice! my_data . some_attr json . loads ( my_data . some_attr ) This is less nice. I have…
When working on a live system, the feedback loop can be pretty tricky. You don't want to entirely suspend a system, but you really want to get the info out of the system. Memory leaks in particular can be a nightmare, because they can take a while to show up and, definitionally, won't survive a process restart. Let's talk about debugging memory leaks. objgraph is a great library to get a birds-eye…
This post summarizes my takeaways from Platform as a Reflection of Values , a very eye-opening talk that helped me understand why I would have the same disagreements over and over with some team members in the past. Let's imagine a universe where you and I are on a small engineering team. I care about testing a lot. I believe the following: Automated tests prevent regressions in production.…
Consider the following snippet of Rust, of a Target struct, along with a builder to go along with it: struct Target { foo : bool , bar : u64 , baz : i64 , } struct Builder { foo : Option < bool > , bar : Option < u64 > , baz : Option < i64 > , } impl Builder { fn new () -> Builder { return Builder { foo : None , bar : None , baz : None , }; } fn foo ( mut self , v : bool ) -> Builder { self . foo…
Disclaimer: as with every performance discussion, nothing beats measuring. Disclaimer 2: Metaphors are great at hiding important distinctions between two situations through massive context flattening. Having said that, metaphor time. You have just graduated from the Very Fancy School With An Important Alumni Network. As a graduation gift, you are given two school alumni contact books: One is a…
The following is something I know in my heart but always take a minute to reconstruct when it comes up, so I am posting it here for posterity. Typescript lets you write type C = A | B . This often feels more ergonomic than when many languages force you to tag the union. C = Either A B , then doing left a or right b , all followed by needing to unwrap the values at the usage site... it feels like…
The other day this snippet of Typescript generated by Github copilot was floating around: type MappedParameterTypes < T extends Parameter [] > = { // Check if the parameter has an 'enum' defined [ P in T [ number ] as P [ "name" ]] : P extends { enum : Array < infer E > } ? E extends string // Ensure the enum values are strings ? P [ "required" ] extends false // Check if the parameter is optional…
Property testing is a very helpful form of testing. At its core, it involves doing the following: Generate a "random" value Test whether some property for that value holds I the property holds, "shrink" the value, to try and simplify it, and see if the test still fails So in practice, you can have some idea that your system validates something like the following: for all ( list : List ) , reversed…
I have been trying to use nushell as my daily driver, and it's pretty cool! It has a terseness that makes people come to POSIX shells, but offers a nicer syntax, just the right amount of type safety for a shell language, and ultimiately feels worth it. But I found a weird perfomance issue while exploring some of the features. When looking at open files, I wanted to get a structured table from the…
Feature flags are a way for us to enable or disable code paths without needing to re-deploy software. In a world without feature flags, you might develop a new feature on a branch, and then when you merge it into main , the next time you deploy the feature goes out. This feature goes out on the release (so you want to time your deploy based on some marketing strategy), and on top of this the…
Node packages love to use carot versioning. read-pkg-up@"^7.0.1 means that this package requires at least 7.0.1 but less than versions 8 or above. This is great in a sense because it means we can get bugfixes for the same major version, without requiring us to chase down each package maintainer to update the requirements file. But let's look at a certain dependency chain: semver @5.7.1…
When first setting up some containers on a new project, you might be using 2 or 3 tools, and decide to just take some "official" image from Docker Hub as a baseline for a container. Official Docker images are pretty great when you have very little you want to configure. They're often more than sufficient for prototyping when it comes to things that are otherwise fidget-y to set up, like a…
tl;dr don't turn on all the checks for a subset of files, check as many files as possible and add checks on a module-by-module basis This post is a bit of a response to some online complaining by Armin Ronacher about mypy. While I agree in the abstract that mypy -s payoff feels less nice than Typescript's, I spent 30 minutes or so looking at Sentry's codebase and felt that Armin's problems are not…
Quines are programs that, when you run, will print out its own source text. You can read this very long article about how to write one in C. Here we're going to do it in Python and skip a lot of the details and theory. You can, of course, "just" read the data from disk: import sys print ( open ( sys . argv [ 0 ], 'r' ) . read ()) This is obviously cheating though. It only works if we call it the…
A long time ago Javascript introduced strict mode. This gave everyone an opportunity to fix some embarassing stuff in how Javascript worked, up to outright removing some functionality that was bad. Things have of course improved a lot, especially with the advent of various transpiling efforts, more serious standardization workflows, and just a lot more eyes on Javascript as a whole by many more…
What I'm going to describe is just derived from how traits themselves work, but despite having now written a decent amount of Rust, I had not really fully absorbed this fact. Imagine you are working with some cell towers owned by various people, in various locations: struct CellTower { id : String ; location : Location ; owner_id : String ; } You might want to do various things with these so you…
There are a lot of simple ideas that float about in programming. Many that are conceptually simple, yet end up being a bit tricky for you to implement. Right until the moment you see how somebody else does it, and you have a more concrete feeling for the simplicity. What follows is an advanced common sense idea that should help you with printing inside loops. In some batch script you might have:…
What does sudo do? Most people have a basic understanding: it lets you temporarily run a command as root (or some other user, based on the options passed in). People will discuss sudo as nice from a security perspective relative to logging in as root, because you are scoping your usage of the root account to what really needs it. And with the default sudo setup you aren't handing out the root…
While working with the same people in the same problem domain for a long time, it's extremely common for people to take shortcuts when describing usage of a technical system. "When the user submits the form, it fails" "When the process runs out of memory and dies, it will automatically restart" "Oh yeah the app is slow right now" "This API is harder for a user to use correctly" There's a general…
(Update: my conjceture has been disproven , at least with current word lists.) Happy New Years everyone! I hope everyone can enter 2022 with some optimism. Everyone is playing Wordle . It is very cool and fun. It is basically Mastermind, the word guessing game, but with a cool interface and a software keyboard that makes it very addicting. On top of this lots of people are now playing Absurdle and…
When you're introduced to Rust's data types you will often be introduced to usage via match : fn divide ( numerator : f64 , denominator : f64 ) -> Option < f64 > { if denominator == 0.0 { None } else { Some ( numerator / denominator ) } } // The return value of the function is an option let result = divide ( 2.0 , 3.0 ); // Pattern match to retrieve the value match result { // The division was…
Here is a mental transcript of a bunch of things I tried to do this morning when trying to figure out how to listen in on a socket and get information from a client. I have never done this in a language with manual memory allocation, and that small detail expanded into many issues. On the server-side, we have a TcpListner , giving us a TcpStream when calling accept() . Remember the canonical…
Whenever I started learning about parsing and writing interpreters, I would get to how to handle comments. The gist of it was simple: just throw them out! You find them in the raw text output as you are generating lexing tokens, and discard them, meaning that later phases won't even see them, let alone have to handle them in any real sense. And this is something that I end up seeing over and over.…
Let's say you have a web app with some database. In your database you have an Invoice model, where you store things you are charging your customers. Your billing system is flexible, so you support setting up tax inclusive or exclusive invoices, as well as tax free invoices! In this model you store it as: #!python class Invoice : ... is_taxable : bool is_tax_inclusive : bool You use this for a…
Developping for the web is fun, Webpack is a marvel of modern enginerering, and when you have a project fully set up with the latest and greatest tools, things work amazingly. But when you have a small, throwaway web projet, Webpack and all that stuff starts to feel a bit like overkill. I'm not that interested in tracking my dependencies explicitly, writing up a package.json , trying ot remember…
Rust has an amazing dbg macro that lets you quickly set up an expression printer that will also put in the source line. It also returns the value of the expression so you can even easily inline the printing when you want to! # ! rust let a = 2 ; let b = dbg !( a * 2 ) + 1 ; // ^ -- prints : [ src / main.rs : 2 ] a * 2 = 4 assert_eq !( b , 5 ); Doing a bunch of Python, I want this in Python as…
Last time I spent some time digging into Pylint, I was able to identify some memory issues, but hadn't really changed the runtime of Pylint. This time around I'm going to see if I can identify something that I can make actually run faster, instead of just using less resources. My Personal Profiling Principles Function f is taking some time. You have the following options: find a way to make all…
Through work I have to deal with a pretty huge codebase, and run Pylint on it during our continuous integration process. It's always been a bit slow, but recently I also noticed that it was consuming a lot of memory, causing OOM failures if we tried to run it too much in parallel. I decided to roll up my sleeves and figure out: What exactly was consuming so much memory? Is there a way to avoid…
I have started playing around with FPGAs, by trying to build a (very simple, 2D) video card. For those who don't know, FPGAs are basically programmable chips letting you build digital circuits without having to go get a bunch of silicon pressed on evey iteration. This lets you do stuff that would be near impossible to do with a general CPU in software, like real-time signals (making it perfect for…
Every couple of months I like to take part in a game jam. It's fun to try and build a thing, and also feel totally fine with never looking at it again afterwards. Usually these happen online, over the course of 72 hours. Probably one of the more famous ones is Ludum Dare . I've done most of these alone, though I have in the past also done jams with one other person. In the beginning of November I…
Remember files? A lot of awesome stuff existed thanks to files being the atom of computation, but it's all somewhat disappeared of late So much work on desktops resolves around files. Your desktop has a bunch of files, and you use programs to work on them. So you have your spreadsheets, images, and other goodies. And you have your tools to operate on them. so you work off your files. They're…
Sometimes I'll find a new website where the content is so entrancing that I want to read through all the archives. But it's pretty time consuming to sit and read everything at once, and I'll always lose track of where I am. Luckily I have a Kindle, a computer, and Python. So I've started throwing together simple e-books with this content. I get the nice auto-bookmarking and screen of the Kindle,…
ORMs have been a huge time saver for society. They line up pretty well with the "enterprise"-y software most of us write. They eliminate a good class of potential errors involved in marshalling data through SQL. The amount of things only expressible in SQL has gone way down over the years. There is a bit of a danger in how ORMs get used. The classy nature of ORMs make us reach for seductively…
Let's say you're building a standard web application, where you have Clients, and Documents for those clients. You want to build a page that lets you browse documents, based off of clients. Something like the following: [ Client A | V ] <- Client Selector --------------------- - Document A <--- List of Client A ' s documents - Document B You decide to load in the list of documents only after a…
So I've been messing with Purescript recently. Purescript is a purely functional language that compiles to Javascript. It includes support for things like object literals that make Javascript interop pain-free. It also has mature tooling, making getting started quite simple. It also has Row Polymorphism . I don't want to exagerate but this is a dream come true when it comes to encoding of best…