computers i guess
2025 Update This post has aged remarkably well from its original release on 2023-12-11. That being said, I have found myself working on build systems full-time in the year and a half since then and have gotten more familiar with the industrial state of affairs. I have recently been working on a large scale deployment of Facebook's Buck2 at my job (not at Facebook). The opinions in this post…
A pattern I've seen happen many times is where people (often new to Nix) overuse Nix in a manner that makes their lives harder, in order to make their systems ostensibly more reproducible and often get the bad side of the tradeoffs inherent in using it. The type of way that this becomes frustrating is, for instance, by inserting Nix evaluations and builds in the iteration loop of changing…
Recently I had a very hard problem on my hands: Lix on macOS wasn't terminating daemons when the client vanished (e.g. if the client got CTRL-C'd or crashed). This is a rather .. weird symptom, since the mechanism for detecting that is very simple. Immediately I suspected there was a kernel bug (or misused system call API) involved, since the fault only happens on macOS and the code was…
Start of Meetup: "hmm, Kane is using nixos channels, that's not good, it's going to gaslight you" 6 hours later: Utterly bamboozled by channels 6.5 hours later: I am no longer using channels - @riking@social.wxcafe.net Nix channels, which, just like Nix, is a name overloaded to mean several things, are an excellent way to confuse and baffle yourself with a NixOS configuration by making…
Although Nix supposedly makes pinning things easy, it really does not seem so from a perspective of looking at other software using pinning: it is not possible to simply write package = "^5.0.1" in some file somewhere and get one package pinned at a specific version. Though this is frustrating, there is a reason for this, and it primarily speaks to how nixpkgs is a Linux distribution and how Nix…
I was making memes in the CTF room until someone told me Nix showed up on a CTF, and well. It doesn't take that much to tempt me. Reproducible Pwning is a challenge written by niko , which involves a NixOS VM you're supposed to root. The build user is not notably privileged. There is a flag in /data which is mounted from the host via some means. That directory is only readable by…
An idea I have lying around is something I am going to call "ca-tracing" for the purposes of this post. The concept is to instrument builds and observe what they actually did, and record that for future iterations such that excess dependencies can be ignored if, even if inputs changed , the instructions are the same and the files actually observed by the build are the same. Implementation…
Inflammatory title out of the way, let's go. I think that Nix flakes have some considerable benefits, such as: Convenient pinning of evaluation-time dependencies Eliminating pointless rebuilds of code by only including tracked files in builds Making Nix code, on average, much more reproducible by pervasive pinning Allegedly caching evaluation Possibly making Nix easier to learn by reducing…
I was debugging an issue with a friend's website that was Safari-only, and a pretty funny issue at that: autoplay videos won't play inside iframes, even if the navigation of the iframe is caused by a user gesture. Turns out it's a likely WebKit bug, but we needed to look at it to find that out. Like most mobile browsers, Safari supports being debugged from a computer. Unfortunately,…
This is a quick post about an issue I debugged in a package build that didn't matter, which may however still be instructive. I was packaging ios-webkit-debug-proxy for NixOS, not having noticed it was already there (oops! I noticed after fixing my build), and I ran into an issue where my build was not working, so I tried building it manually, and it worked. Wat. Here is the package.nix I was…
This is a quick post to write down the method that worked for me to acquire and pull apart a Dell UEFI image to get the executables out, and how to poke at ACPI. The specific issue I was curious about and trying to find more information about is the rtsx card reader NVMe regression on the Dell XPS 15 9560, which is the laptop I use. The regression was caused by the card reader driver no longer…
Slides here: https://jade.fyi/nixcon2023/slides/
I have been using an unmaintained fork of prezto since 6 years ago, which has been mostly completely fine. However, one day (and possibly due to my own actions), it started having severe startup performance issues on my NixOS system one day, and enough was enough: I will finally figure out how zsh works. It would be unfair to blame prezto for the startup performance issues, since the version in…
Sometimes I get jealous of web developers for having real network debugging tools for watching HTTP traffic, as (primarily) a native developer. Not anymore: I have their tools now. You can have them too. A while ago I was debugging an issue with the Rust OpenTelemetry library on my computer and it was immensely frustrated by not being able to get the actual requests out of it. Eventually it…
As part of the third year design studio class for Computer Engineering at UBC, I worked on a group project using a FPGA, electronics, and other pieces. We had to pick a project idea and then build it in a 4 month term as a group of 3 or 4 (we were 3). The group I was in built a robot called Meowzor that points a laser in front of a cat, using an object detection model (MediaPipe) to find where the…
Modern cameras put a lot of metadata into images: GPS location Device model Camera software version Subject distance Facing direction Colour profiles Time, with timezone Reasonable camera stuff like ISO, shutter speed, flash usage, focal length This is mostly good actually, since it is useful to you as the photographer. I've very frequently found use for photo geolocation of old photos.…
I have been working on a school project, which uses Intel FPGAs and the proprietary Intel Quartus toolchain. One of the components in the design we're building is a Nios II embedded processor, which is Intel's weird proprietary soft-core. In newer versions of the highest-end SKU of Quartus, Intel has finally made a RISC-V core that integrates similarly, but we don't have that. There…
Nix has a feature called "import from derivation", which is sometimes called "such a nice foot gun" (grahamc, 2022). I can't argue with its usefulness; it lets Nix do amazing things that can't be accomplished any other way, and avoid pointlessly checking generated files into git. However, it has a dirty secret: it can atrociously slow down your builds. The essence of this feature is that…
the first thing our new hire did was fix a bug that's been bugging him forever as a user prior to joining. he then breathed a sigh of relief and submitted his two weeks' notice. wtf?? Tweet by @swaglord__420 on Mar 30, 2021 In celebration of the fixing of the notorious 4 year old VSCode bug which practically everyone I know has seethed about where diagnostics show below pages of function…
Well, that sure was a thing. It was more than a thing: it was a really good thing. NixCon 2022 was my first in-person conference, and therefore also my first time speaking at an in-person conference. I understand better why people want these. It's really nice to be around The People For The Thing and meet new people (whether they are wearing cat ears or otherwise). I met a whole bunch of…
Dealing with PDFs is something I do every day as someone working in software, especially given that I tend toward both research and lower-level work where papers and datasheets rule. I think that the humble PDF is one of my favourite file formats besides text: You can give someone one and it will work Vectors work great in it Old files also just work Anywhere in the continuum between…
Note : I just rewrote the syntax highlighter for my site's generator, Zola, to use tree-sitter, and there are probably some deficiencies. Let me know if there's anything going terribly wrong. I was reading a beta copy of Production Haskell by Matt Parsons again, and a section stood out to me in the type level programming chapter (something I am trying to get better at): "Value…
I delivered a talk "Debugging Closure Size Graphically" at NixCon 2022. This blog post is intended as a textual version of the talk although it is not a script. Here are links to the live version: Livestream video (a better recording with slides is coming soon) Slides Tools: nix-closure-graph:…
I found something mysterious while working on a large Haskell codebase: there was a Cabal target called test:test-dev , containing test and src in source-dirs . For a while, I didn't realise what it's for, and it seemed like the kind of thing that someone goes and creates for some old use case. Now I have put this into at least three other codebases because it fixes my problems. What…
A problem I sometimes have is dealing with a bunch of independently maintained but very interdependent libraries, for which I would like proper IDE functionality cross-project. Another problem I sometimes have is that I want to make a contribution to some library but setting up an IDE for it is a pain in the neck. YOLO method (putting it all in a workspace with your app and not telling Nix) Often…
I work in a thoroughly Nix-based environment, and sometimes I need to test NixOS things, but I have a Mac at work. So, time to get a NixOS VM going on the M1, I guess! This was a partially-planned adventure. People I know have been saying good things about the (mostly) qemu wrapper UTM ( also on GitHub here ), so that was the tool of choice. I kind of didn't realize there were normal…
When writing, debugging, or otherwise working with Nix expressions, it is often useful to run a build part of the way through, or to run it manually in a shell. For instance, perhaps you want to test some development version of a tool against a package and want to iterate quickly. Or, a build is broken and you want to look at it more closely than nix-build --keep-failed makes convenient. Most…
Note A previous version of this article unfairly implied that Firefox has better SVG standards conformance than Inkscape. The situation is more complicated than that and includes the standards being cut down to match stagnant browser implementations, while there are also some SVG 2.0 features missing in Inkscape. Nobody is perfect here. My partner has a brilliant poster they made of the backing of…
Say you have to contribute to some project run by Unix bros who want the 90s back, so they don't use GitHub or its many self-hosted alternatives to accept pull requests, and you need to email them a patch. I would find it entirely understandable to want to get a quill pen and write a letter on paper for fun instead of dealing with mailing lists and emailing a patch. Unfortunately, sometimes…
I have wasted a lot of time debugging multi-process systems, especially those that are some native program with a shell startup script. Although gdb claims to support child processes with set follow-fork-mode ( docs ), in practice, this is extremely painful since gdb needs to be told, at each fork, which way it should go. To deal with these, I've done such hacks as writing wrapper scripts for…
In this year's crop of classes, I am using some Fun™ software in obsolete versions that would work properly if I would just use Windows like a reasonable person. That was not an acceptable choice for me, so I got cracking on the experience that is trying to get old proprietary software running on Linux. This procedure was tested working on Quartus 16.1 and 18.1. It is entirely unnecessary for…
nixpkgs has good built in support for building Docker images, but there is a significant amount of nuance to using it effectively and making small images. It is very easy to accidentally put more stuff in than you want, and this post will document how to Not Do That. A useful companion to this post is the source code to the nixpkgs Docker module, at…
Improving and automating my workflow is something I have put considerable investment into, and I would like to share what I can. This is part of a series of posts about my development workflow. You can read the other installments here: Docs I have a curse: I know how to program and I run Linux, so I naturally tend to fix things when I see them, which results in significant yak shaving, but also…
A while ago I read a tweet that reminded me I felt guilty about the state of my ssh key security and wanted to do something about it, eventually. Well, my watch reads "eventually", so let's do it. My workstation is a modern AMD system, which has a firmware based TPM 2.0 implementation available "for free", which sounded pretty nice, so let's use it. This guide is partially sourced from…
Recently, I was working on a tool that incidentally makes ephemeral units on the systemd user instance to do what one would use atd for on an older system. As part of working on that, I needed to figure out what RPC calls I had to do to add the unit. Attempts to use dbus-monitor or busctl monitor on the session bus failed to find any traffic. I investigated this by strace ing the execution to find…
TL;DR: check out my GNU docs builds at https://docs.jade.fyi/gnu They uniquely have: Limited width/better-font-size CSS I wrote One big page for each tool!! Send me an email if you want some page that's not in there, I'll see what I can do now that I have the tools built. I have recently adopted the practice of locally building and serving HTML documentation for…
Hi! Well, it's about a year late to be posting this, but I'm transgender (and many other labels, which I have elided). In late 2019, I had a hell of a long weekend where I reconsidered a bunch of dusty memories I had forgotten about or repressed. I also rewrote the website while contemplating this . At the time, though, I was not very confident in how I wanted to present myself in…
It is a poorly guarded secret that the most effective way to get documentation on nixpkgs or nix functions is to read the source code. The hard part of that is finding the right source code. I'll document a few static and dynamic analysis strategies to find the documentation or definition of things in nixpkgs going from trivial to some effort. Simple These work on functions that have no…
I'm going to try a new style of blog post for this one, more of a lab notebook style. Expect normal posts for more interesting topics than fixing build systems. ISSUE: workspaces interact very poorly with targets BUGS: https://github.com/rust-lang/cargo/issues/7004 You can't make Cargo use a separate target per workspace crate. Thus, workspace wide builds…
Hello! The full content on this site is available by RSS at https://jade.fyi/rss.xml . I can make no promises as to how fabulously RSS readers will render the full post contents without some kind of styling, but they are included. If you have any further requests on how I can make this site work better for you, please email me at the address on the About page or file an issue on the…
I recently was brutally nerdsniped into developing a strange Rust library that turns prints into uwu-speak . I briefly considered writing a proc macro but that was far too memory safe. It's doing-bad-things-to-binaries time! (followed shortly by uwu time~!!) I am going to use Linux because it's the platform I'm most comfortable doing terrible things to. I thought of a few strategies…
This post was inspired by a hypothetical closed source piece of software from a hardware vendor, written in Java, which has unusable font rendering that makes it inaccessible for me, but I need to use it for class, so what am I to do? I want to write evil LD_PRELOAD hacks but it's probably easier to patch the program itself, so that's what we're going to do. I use IntelliJ IDEA for…
In my Google CTF entry for writeonly this year , I wrote my first stage shellcode in C, which was somewhat novel in and of itself, as it seemed like few people were willing to brave linker scripts to be able to write shellcode in C. My hubris does not stop at C, however, and the crab language seemed well suited for a port. Source code here As with the previous C implementation, what we are doing…
Template Haskell powers a lot of really neat functionality in Yesod and friends, but sometimes it can be broken. I'm writing this post to collect all the info learned about GHC and Cabal from an unpleasant debugging session in one place. I was tracking down a problem causing my work project to not build on a newer GHC version ( spoiler: it was this persistent bug ) and hit a brick wall when…
I participated in the 2020 Google CTF on the UBC CTF team Maple Bacon . Without their help, I would have probably given up out of frustration. Special thanks to Robert and Filip who put up with my many questions and swearing at the computer. All the files for my solution are available on my GitHub . I chose to do this challenge as nobody else on my team was working on it and it looked fairly…
I'm writing this post because I work every day in WSL 2 on my main computer and I feel it might be useful to those trying to get a productive setup running. I use Arch Linux inside WSL, with the ArchWSL project . Arch is used since it's what I've installed on my other computers, and is compelling for the same reasons: up to date packages, reliable, and is easy to package things for.…
UPDATE : August 28, 2021 - There is now much improved official documentation of a lot of this material. This post is not obsolete as it covers GHCJS. UPDATE : March 13 2021 - rewrote a fair amount of the post As part of my job working on an open source logic learning management system , I picked up a Haskell codebase that was rather hard to build. This was problematic for new contributors getting…
I recently acquired a new keyboard, which was a Whole Thing (tm) as I ordered it right at the end of Chinese New Year's, in time for the entire country to be locked down for COVID-19 reasons so it ended up turning up yesterday, three months later. It's a KBDFans DZ60 in an extremely normal layout, with Kailh Box Brown switches. I bought it to replace my Unicomp keyboard which was mostly…
As part of developing this new version of this site, I've needed to mess with nginx a lot to switch from Ghost to Gatsby, especially when related to hosting files out of multiple directories. Specifically, this site is deployed by rsync ing the production version of the site onto the server hosting it. I want to be able to use --delete to get rid of any old files for reliability reasons…
I just rewrote the formerly-Ghost blog in Gatsby, which may improve performance, will definitely improve image loading, and will substantially simplify maintenance. A significant motivation was that maintaining a database is a pain, especially on Arch Linux where I have to do migrations on a somewhat unplanned schedule on pace with the release schedule of the database system. Another further…