Recent content on Carlo Hämäläinen
Back in the old days, people used to leave comments on blogs. Imagine such a time. One time a guy even sent me a bottle of his homemade wine in appreciation for a post I made about getting MinionPro fonts to work in LaTeX! My own blog has migrated from Wordpress (self-hosted), to Haskell/Yesod (custom framework, self-hosted), and finally to Github static pages. Wordpress handled comments, so that…
My current setup: Small townhouse in Brisbane. 3.9kW solar array (ugh small roof). Sungrow SH5.0RS inverter. Sungrow SBR096 home battery (9.6kWh). BYD Atto3 Premo. AGL have a Night Saver EV Plan with very cheap electricity from midnight to 6am each day (at my location, only 8c/kWh). I mostly charge the car in that time period. Sungrow provide the iSolarCloud app/site that provides control of the…
Full source code and talk: https://github.com/carlohamalainen/bfpg-2026-05-12-datastar-haskell-sdk
The Question Link to heading When we have many data structures that contain the same large value, does GHC preserve sharing to save memory? For example, if we create 100 Foo values that each contain the same large HashSet , will GHC be smart enough to share that single HashSet in memory, or will it create 100 separate copies? This minimal example demonstrates that GHC does preserve sharing when we…
Performance surprises are everywhere - even in the simplest code 🚀 At my old bank job, I discovered something eye-opening: for some requests, our code spent more time logging than doing the actual mathematical calculations and data loading combined. The culprit? Our timestamp formatting was using a custom format string, and our internal datetime library was invoking an expensive parser to read…
Software estimation is hard. Some teams rely on story points. Others prefer function points or t-shirt sizing. I think the current feature I’m working on is an outlandish lamington and a flat white. Mmm #softwareengineering #estimation #agile #coffee Crossposted at LinkedIn
Have you ever typed $ sudo chmod -R 400 * and then wondered which directory you were in? It happens to the best of us. I figured this would be a good opportunity to try DuckDB DALL·E rendering of the situation First, ask Claude 3.5 Sonnet to write a Go program to traverse a path, taking note of each file or directory’s permissions in octal format, and write to Parquet files in chunks of…
Setting up TLS with Caddy on a Digital Ocean droplet is so easy with the Caddy DNS plugin . I set up a new droplet with Tailscale and found the challenge kept failing with an error could not determine zone for domain : Aug 08 11:04:37 ubuntu caddy[10197]: {'level':'error','ts':1723115077.6617424,'logger':'tls','msg':'job failed', 'error':'foo.example.com: obtaining certificate: [foo.example.com]…
The goal is to deploy automatic TLS certificates using cert-manager , with certificates supplied by Let’s Encrypt . I’ve tried to produce a minimal set of files from Digital Ocean’s Kubernetes Starter Kit - 03 nginx . I had trouble with that guide due to some strange timeout issues; bumping the version of cert-manager seemed to help (see my PR ). I also changed to a…
I migrated my blog from Jekyll to Hugo , mainly because I have no experience with Ruby and don’t have the time to stay up to date with another language and framework. I did most of the migration in a mega-commit . I went with the Hugo-Coder theme, only making one minor tweak for the width of content : . container { margin : 1 rem auto ; max-width : 120 rem ; /* max-width: 90rem; */ } I only…
Heatmaps are an excellent way to visualise real-time system performance, as they provide a concise and intuitive representation of response time distributions across multiple dimensions. Some of these dimensions include: Time: a heatmap can be plotted over different time intervals to find temporal patterns in system performance. System components: by filtering on different subsystems, we can see…
This is a fun talk about what goes on before your program starts executing main : Matt Godbolt mentioned the oddly named “bss” section. The book Advanced Programming in the UNIX® Environment, Third Edition gives the historical reason: Uninitialized data segment, often called the “bss” segment, named after an ancient assembler operator that stood for “block started by…
Wonderful visualisations in this talk : ScyllaDB’s CTO Avi Kivity dives into how high performance distributed systems such as modern databases can make best, most efficient use of fast NVMe storage.
A normal map lets us look up items based on an exact match for the key, but here is a situation where we want to match on a superset of the search key. We generate some expensive objects related to sets of tags. For example: { "foo", "bar", "baz" } => #{expensive object 0} { "apple", "cat" } => #{expensive object 1} { "dog" } => #{expensive object 2} Example lookups:
I have no idea why, but a week ago my AirPods suddenly became far too quiet on Linux (Debian buster). Changing volumes in pavucontrol didn’t help, nor did tweaks that people suggested like setting --noplugin=avrcp nor --plugin=a2dp . The issue seems to be that Bluetooth headphones have an internal volume level. When connected to an iPhone this volume is set appropriately but something goes…
In the old days fitting 26" mountain bike tyres was so easy. I could usually get the tyre off with one plastic tyre lever and tyres would often mount without a lever at all. These days people like to run tubeless so tolerances between rim and tyre sizes have become quite small and many people are finding it harder to mount tyres . Personally, I’m still running tubes. I nearly gave up trying…
Profiling tools like gprof tell us about where a program spends the most time but these places aren’t necessarily the best candidates for optimisation. Also, we have no idea of how much overall improvement we would see for a certain amount of optimisation. This problem gets harder for multithreaded or distributed systems. Here’s a system for image classification (taken from this great…
The end goal here is to have Unity3D and Visual Studio Code installed on a Debian 10 Linux system with Intellisense completions working for Unity code. You should be able to type the start of a Unity name like GameObject and see the completion: I mostly followed @nosuchstudio’s guide for Ubuntu 18.04 LTS but added some extra details and confirmed Debian 10 compatability. There are four…
(Disclaimer: I’m not recommending the use of R, this question just came up in some of Nadiah’s work. I suggest Seaborn .) If x is a dataframe in R, then colnames(x) gives you the column names: > colnames ( x ) [1] 'Column1' 'Column2' 'Column3' R supports “complex assignment”, so you can do this: > colnames ( x ) <- c ( 'a' , 'b' , 'c' ) > colnames ( x ) [1] 'a' 'b' 'c' and…
Update 2024-02: I wouldn’t bother doing any of this, I’d just use vscode-remote in VS Code. Easy! Vim and Neovim come with the netrw plugin for editing remote files. The way I use this is to write a list of frequently used files in my scratch file for a project, and go to edit the file by putting the cursor over the line and hitting gf :…
BeautifulSoup makes it easy to quickly scrape content from web pages. Here are two examples. Electricity prices from Tocom: https://www.tocom.or.jp/market/kobetu/east_base_elec.html The page has three blocks (Current, Night, and Day sessions). Each block is under a h3 , with the first table providing the session name and date, and the second table provides the prices. The first table is a bare…
Here is a well-known interview/code golf question : a knight is placed on a chess board. The knight chooses from its 8 possible moves uniformly at random. When it steps off the board it doesn’t move anymore. What is the probability that the knight is still on the board after $n$ steps? We could calculate this directly but it’s more interesting to frame it as a Markov chain. Calculation…
The farmer-chicken-fox puzzle goes something like this: a farmer is at a shop, having bought a chicken, fox, and a bag of corn. The farmer would like to get to her house on the other side of the river using a small boat. For some reason she can take at most one item at a time. If the chicken is left alone with the corn, it will eat the corn. If the fox is left alone with the chicken, it will eat…
Many business problems boil down to reading data from somewhere, transforming it, and writing it somewhere else. We could implement the transformations in code, but non-technical users might like to see and edit the rules without having to deploy a new build. Here’s an example rule: Read the value at https://example.com/foo/bar/x , refer to it as val . Return 10*(1.0/val) . Or, a more…
I’ve been using my Glacier Push utility for a while now and it has been working well. But recently I noticed that on really large files the initial memory usage spiked, and then fell off. I suspected that the treehash calculation (implemented in Haskell) was not efficient so I re-implemented it in plain C. The first thing was to write a utility to calculate the SHA256 of a buffer.…
I can never remember this. https://bitbucket.org/delx/webdl git clone https://bitbucket.org/delx/webdl cd webdl virtualenv --python python3 .virtualenv . .virtualenv/bin/activate pip install -r requirements.txt Usage: . .virtualenv/bin/activate ./grabber.py
Edit 2019-01-12: fixed here: numpy/pull/11977 Nadiah ran into an apparent memory leak in SciPy’s hypergeom distribution. Here is a minimal example. Running this code results in unbounded memory use, looking like a memory leak: from scipy.stats import hypergeom while True : x = hypergeom ( 100 , 30 , 40 ) . cdf ( 3 ) It turns out that this isn’t really a memory leak but rather a problem…
Using a modern GHC compiler, how much memory would this program use? x <- readFile 'foo' x ` deepseq ` print () Linear in the size of foo? Or something else? Turns out, for the default readFile from the Prelude, the answer is about 40 times the size of the input file. The default Haskell strings take 5 words per character, so on a 64bit machine this is 40 5*8 = 40 bytes per character. The list of…
Property based testing is a great way to improve code quality since it captures logical properties of your system. Instead of writing test cases by hand, you capture logical relationships and then let the test framework generate hundreds or thousands of examples for you. For a general introduction to property based testing (language-independent), try this YOW! Night talk Property Based Testing…
Consider an option with payout on an underlying which is a cumulative sum. For example, a construction company might want to mitigate the risk of rainfall causing delays in a project. The construction company could buy a call option on the cumulative sum of rainfall not exceeding 180mm on any 3-day period. Further, we restrict this so that one day can only be used in one payoff. If the cumulative…
Here is a small Haskell package for pushing files to Amazon Glacier: https://github.com/carlohamalainen/glacier-push . It uses Brendan Hay’s amazonka API , in particular amazonka-glacier . One thing that I couldn’t find in amazonka was a way to calculate the tree hash of a file. The Glacier API needs this for each part that is uploaded as well as the whole file. Amazon explains how to…
My Boardman Team TXC 650b hardtail mountain bike has an FSA crankset and bottom bracket. After a year and a half the bottom bracket got quite rough so I decided to swap it out with a Hope Hollowtech II bottom bracket. I couldn’t find much online about this bottom bracket. Markings include: “FSA B3164 MegaExo 24mm MS185” and “BC1.37” x 24T". I replaced it with this…
I have a pull request to merge ghc-imported-from into ghc-mod . The main benefit of being part of ghc-mod is that I don’t have to duplicate ghc-mod’s infrastructure for handling sandboxes, GHC options, interfaces to other build tools like Stack, and compatibility with more versions of GHC. The pull request is still under review, so until then you can try it out by cloning the…
I plugged in a D-Link DUB-1312 to my laptop running Ubuntu Zesty but Network Manager said that the interface was “not managed”. The fix, found here , is to remove the contents of one file. Better to save the original file and touch an empty one: $ sudo mv /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf{,_ORIGINAL} $ sudo touch…
2024-03-13 These days we have an excellent ecosystem with Grafana Loki, Promtail, Tempo, and friends A while ago I wrote about how to set up a structured logging service using PostgreSQL . AWS now makes it possible to have the same functionality (plus more) in the “serverless” style. For background on the idea of serverless architecture, watch this talk: GOTO 2017 • Serverless: the…
I had a situation, when converting old blog posts to WordPress, where I wanted to strip all the extra info on the pre tags. For example this: <pre class="brush: plain; title: ; notranslate" title=""><pre><code><span style="">></span> <span style="color: blue; font-weight: bold;">import</span><span style="">Data</span><span style="">.</span><span style="">Char</span> would turn into: > import…
Short note on Data.Proxy based on this Stackoverflow answer . First, a few imports: {-# LANGUAGE RankNTypes #-} {-# LANGUAGE ScopedTypeVariables #-} module Proxy where import Data.Proxy import Text.Read Suppose we want to check if some fuzzy real world data can be read as certain concrete types. We could write a few helper functions using readMaybe : readableAsInt :: String -> Bool readableAsInt s…
I have a pull request to merge ghc-imported-from into ghc-mod . The main benefit of being part of ghc-mod is that I don’t have to duplicate ghc-mod’s infrastructure for handling sandboxes, GHC options, interfaces to other build tools like Stack, and compatibility with more versions of GHC. The pull request is still under review, so until then you can try it out by cloning the…
This blog post shows how to convert a standard Raspbian installation to full disk encryption. The encryption passphrase can be entered at the physical console or via a dropbear ssh session. I mainly follow the Offensive Security guide . What you need: Raspberry Pi. Laptop with a microSD card slot. I used my X1 Carbon running Ubuntu xenial (amd64). First, install Raspbian . With a 32Gb microSD card…
Over the xmas holidays I set up a security camera for my parents. I used a Foscam FI9805E IP camera with a Raspberry Pi (model 3) as the server. An old 1Tb USB hard drive provides ample storage on the Pi. Here is the layout: The camera supports power over ethernet but it comes with an AC adaptor so you can use a normal ethernet setup as I have done.
I recently got two YubiKeys to try out another form of 2FA and to see how they work with my PGP setup (Enigmail and Thunderbird). I followed ankitrasto’s guide ( part 1 and part 2 ) to move a key to the YubiKey. I then exported my public key with gpg2 -a --export carlo@carlo-hamalainen.net and sent it to a friend. He replied with the reasonable question: why didn’t the fingerprint…
I was looking at an old post of mine on free monads and wondered what it would look like using operational . Here we go! (Literate Haskell source for this blog post is here .) First, some imports. We use GADTs for our instruction type. {-# LANGUAGE GADTs #-} module Note where import Control.Monad import Control.Monad.Operational import Test.QuickCheck import qualified Data.Map as M We want to…
I was reading some notes from QUT about using spectral graph theory to partition a graph using eigenvalues of unnormalised Laplacian. Their Matlab code looks like this: % Form W = Gaussian distribution based on distances W = zeros ( 100 , 100 ); D = zeros ( 100 , 100 ); sigma = 2 ; N = 100 ; for i = 1 : N for j = 1 : N if ( j ~= i ) % Calculate the distance between two points dist = norm ([ A ( i…
This post has some errors; see ghc-imported-from–ghc-mod-march-2017 for the latest instructions. I have a pull request to merge ghc-imported-from into ghc-mod . The main benefit of being part of ghc-mod is that I don’t have to duplicate ghc-mod’s infrastructure for handling sandboxes, GHC options, interfaces to other build tools like Stack, and compatibility with more versions of…
When I post a series of photos to a personal blog I find myself editing HTML in Vim and switching back and forth to a browser to see if I have written comments in the right places and ordered the photos correctly. I could use a HTML editor to do this, but why not try FRP with Haskell? :) Apparently I sort of use FRP at work so trying out Reflex wasn’t too much of a leap.
Applicative forms in Yesod are nifty but they don’t let you customise layout, CSS, and so on. I had this form for comments on my blog : commentFormOLD :: EntryId -> Form Comment commentFormOLD entryId = renderDivs $ Comment pure entryId lift ( liftIO getCurrentTime ) areq textField ( fieldSettingsLabel MsgCommentName ) Nothing aopt emailField ( fieldSettingsLabel MsgCommentEmail ) Nothing…