Personal blog about hardware, software, data and other interesting things.
I recently realized I had to delete my YouTube app. I was on the usual journey one goes through, some call it a rabbit hole; Luxury Watches, Gold Mining (sluicing washing crushing refining smelting), gold extraction from discarded items, professional mining (so fun to watch shotcrete being applied), professional tournament poker, board game reviews, board game let’s play, poker drama, chess and…
I recently discovered Textual, a Python library for building terminal UI applications and thought it would be fun to figure out how to package an application that is built with textual for Linux, macOS and Windows. This is what I came up with. I decided that in addition to Textual , I wanted support for for numpy (to prove that Python C extensions work), runfiles, so files that are shipped with…
Git has been around since 2005 and has become the de-facto standard for controlling versions of source code. I recently asked myself if Git will continue to dominate and if innovation is possible or even necessary. I will go through some of the major drawbacks of Git, that could motivate innovation, list Git alternatives and conclude with my personal thoughts on the future of version control. If…
Monorepos are great. They implicitly enforce the One Version rule, which means that for any dependency, there is only one version used across the code-base. This can mean that there is only one set of lock files for the entire repository (think single requirements.lock , Cargo.lock etc. for the repo) as well as automated integration testing of all of the latest code. Cosmos by Ilya Chashnik.…
Recently I came across the concept of "messing up as much as possible" so one can learn as much as possible. I first encountered it when following and reading about SpaceX Starship April 20 2023 orbital test flight. Quite a few things went wrong during that test flight: three engines did not work during liftoff more engines failed during flight the ability to steer the rocket was lost, with…
The most significant art exhibit I ever attended to date was the 2013 remake of a 1969 art show in Bern, Switzerland called "Live In Your Head: When Attitudes Become Form" where Harald Szeemann "wanted to show everything that we had not yet seen, everything that came after 'Pop Art' and 'Minimal Art'" (Grégoire Muller, exhibition catalog). There are many memorable pieces of art from that show but…
Recently I've been interested in generating "3D" acoustic soundscapes; I looked into tools that allow creation and rendering of such immersive audio scenes. Excerpt of the impact of anthropogenic activity across the contiguous US on sound pressure levels by National Park Service Natural Sounds and Night Skies Division First and foremost I looked into what tools exist in this space. Here is an…
I never learned about containers. The OS-level virtualization kind. In this article I'm documenting my journey of trying to understand what it is all about and hopefully getting excited about virtualization. I think the two main reasons why I never learned containers are: When containers were introduced I wrote software that heavily relied on hardware - in particular graphics processing units…
I recently quit my job at Butterfly Network and am now spending some time writing code I am excited for, covering topics I want to learn more about. It is an incredible feeling to start with a blank source-code file. It can also be somewhat intimidating - many decisions must be made. But in my case it was quite liberating as I did not care about technical debt or making the perfect decision. I am…
As a computer scientists working on image processing algorithms I know about the frustrating process of hunting bugs on massively-parallel architectures like GPUs. Sometimes those bugs can produce completely incorrect images or sequences of images but with a certain aesthetic appeal. The art world, with its superior ability to name things (compared to computer science), calls these accidents…
Inspired by an old but brilliant Clifford Stoll video and by my desire to get back into running experiments and reporting the results, I recently set out to measure the speed of sound using a set of Roland CS-10EM binaural microphones and a Zoom H2n I borrowed. Concorde, image by Tom Močička In the experimental setup I place the 2 microphones 0.8 meters apart in a large room (6x6 meters), connect…
As sometimes is the case I'm a bit late to the game when it comes to new technologies. This has the effect that when I try out those new technologies I get excited in the same way folks got excited for it months or years ago. One of those technologies is Serverless Computing . I got the chance to use one implementation, Google Cloud Functions to create a webhook endpoint for the excellent Flexport…
Over the last 3 years I read and listened to a number of books. Some of them stood out to me for one reason or another. I am listing those books here. The list serves the purpose of reminding myself of the interesting tidbits in the book, why I enjoyed it and why I might want to read it/listen to it again. I tried to categorize them but I don't think that is particularly helpful - it is also hard.…
At a high level backfilling in Airflow is a mechanism that allows (re-)execution of Airflow DAG instances for a specific time interval. It is a useful but inherently fragile and brittle process in Airflow. Common use-cases for backfilling data are: the data pipeline code changed (a bugfix for example) and the output data should be refreshed using the new pipeline code an external service (an SFTP…
If you ever wrote software or designed hardware in a regulated industry, it is very possible that you interfaced within a Quality Management System (QMS). I personally have seen a couple of different incarnations of such a system: from paper-based with wet-ink signatures to fully database driven and smart-card sign-off. Especially at smaller companies, paper-based systems are common. Templates,…
A common gotcha when working with airflow is to determine when a DAG is going to execute, and what the execution date-time is going to be when it executes. Especially if a DAG relies on processing data for a given execution date-time only to ensure idempotency. The best way to think about this is not in dates or timestamps but time-spans. An Airflow DAG run is going to cover a times-span - i.e. it…
This is a new blog that will replace my old echelon blog. I will attempt to move old content over to this blog eventually.
During the emergence of electronic communication, the two scientists/engineers Carl Friedrich Gauss and Wilhelm Eduard Weber built the first electrical communications line in Goettingen, Germany. They set up a two kilometer two wire copper line between Gauss' workplace – the Astronomical Observatory – and Weber’s laboratory in the physics department of the Goettingen University. It is unknown what…