Highly irregular postings about technology and life
I owe much of my career to Python and I used to choose Python for everything ... but now I don't. I gave a talk about this topic at the July 2026 Christchurch Python NZ meetup , a risky move for a Python meetup. This talk starts with my early adoption of Python and the subsequent decades of using it for almost everything. It then moves into why Python isn't always the best choice and…
It used to be that if you went down the standalone window manager / compositor path on Linux ( i3 , awesome , sway etc) you'd have to cobble together a panel, screen locking, notifications etc using a bunch of disparate tools. This takes a non-trivial amount of time to get right often involving deep dives into configs and the creation of custom shell script helpers. While this can be fun…
Your contacts data is valuable. Years of accumulated email addresses, phone numbers, and personal information shouldn't be locked away in someone else's cloud service. Even with a trusted provider like Fastmail , having a local backup and the ability to work with your contacts offline is important. This is where vdirsyncer comes in. It's a command-line tool that synchronizes…
I presented a short talk tonight at the Christchurch Python meetup about the Python standard library's excellent pathlib module and why you should use it. The slides are available here . This kicked off the evening before Bevan Arp's fantastic talk about becoming a better developer .
I presented recently at the Christchurch Python meetup about a variety of tools, techniques and tricks I use to be productive as a software developer. Some of these are Python specific while others some were more general. My talk was followed up by an exciting impromptu demonstration by Brian Thorne where he showed how he uses coding agents like Claude Code to accelerate his development workflow.…
A few weeks ago, I had the opportunity to present at the April 2025 Christchurch Python meetup . My talk focussed on modern approaches to storing and retrieving tabular and multi-dimensional data, leveraging cost-effective storage systems and decoupling storage from compute. Some topics covered include: the Parquet file format how columnar storage works querying Parquet files with DuckDB and…
Do you like Python? Do you like games? Are you a fan of autonomously controlled tanks? Then PyBot Arena might be just the thing for you! PyBot Arena is a game engine where participants implement algorithms - in Python of course - for controlling tanks which are then pitted against each other. I created the game for the Python NZ Christchurch meetup as a fun way to end the year, with creative and…
It's been a month of meetup talks for me. The third and final one was about Python's early history starting right from when Guido first started working on it in December 1989 until around verson 2.3 when Python was really taking off. Inspirations for the language and various key features are discussed and an obligatory photo of a very young Guido Van Rossum is included too! This talk was…
Two out of three talks this month done! I was honoured to be one of the speakers at the From virtual envs to my reality session of the Python NZ Online meetup this evening. The session had four speakers covering various approaches to managing Python versions, virtual environments and project dependencies. My talk was about Poetry , positioning it within the wider ecosystem and discussing some of…
I gave a presentation today at the Christchurch Rust meetup that introduced the widely used Axum web framework. The talk covered what Axum is, what it isn't, and some common usage patterns. Brian Thorne followed with an excellent talk about logging, tracing and metrics in Rust. Thanks to everyone who attended for your attention, questions and discussion. The slides are available here .…
Some time ago, I was looking for an idea for a small Rust project to help me get some practical experience with WebAssembly (wasm). I wanted to do something fun and whimsical and when I came across the Rust noise crate I knew I was on to something. After much learning, iteration and procrastination, I present to you Faux Maps , a generator of random, cutesy maps. Through the magic of WebAssembly,…
I've recently become quite enamoured with Logseq . It's a modern, open source notes app and outliner with some unique and powerful features. At home I use it to keep track of personal notes and all kinds of random data, and at work it's become my personal knowledge dump and task management system. Logseq is very flexible and capable out of the box but it also has a plugin mechanism…
I gave a talk tonight at the Christchurch Python meetup titled "Security For Python Developers". It touched on general security issues from a Python perspective such as typo-squatting, and very Python specific issues such as the dark side of the pickle . The slides are available here . Thanks to everyone who came along and for the interesting digressions during and after the talk.
I have an Intel NUC that I've been using for monitoring and backups on my home network. It lives in the corner of a living room and the noise of the fan has been surprisingly and annoyingly loud, even when the machine is idle. On other machines, I've had success using fancontrol to control fan speeds using hardware temperature sensors on the CPU and motherboard so I tried that first. It…
IMAPClient 3.0.0 is out! The big news here is that support for Python 2 - and older Python 3 versions - has been dropped. IMAPClient now support Python 3.7 and later. This allows for the removal of various compatibility shims, expands the set of Python language features that can be used and reduces testing load. Dropping support for older Python versions has also opened up the possibility for type…
I gave a presentation today for the Christchurch Python meetup that talks about why pytest is so such a fantastic testing framework and how to approach common testing tasks and problems with it. Some potential areas of confusion around pytest's "magic" were raised by the audience which led to some interesting unprompted discussions. The slides are available here .
I gave a talk this week for the Christchurch Python meetup that went into how iteration works in Python in some detail. Iterators, iterables, generator functions, list expressions and the various language protocols involved with iteration in Python were all covered. The while loop even gets brief mention. The slides are available here .
Mike Kittridge and I ran a talk last week for the Christchurch Python meetup that covered a large assortment of ways that Python programs can store data and exchange data with other programs. We covered the ins and outs of binary files, text files, buffering, text encoding, newline ending, mmap, pickle, JSON and friends, MessagePack, Protobuf, tabular data formats, multi-dimensional data formats…
The release notes for Go 1.17 mention an interesting change in the Go compiler: function arguments and return values will now be passed using registers instead of the stack . The proposal document for the feature mentions an expected 5-10% throughput improvement across a range of applications which is significant, especially for no effort on the developers part aside from recompiling with a newer…
I gave an online (COVID lockdown) presentation to the Christchurch Python Meetup tonight that covered a host of new features that were added to Python in versions 3.8 and 3.9. While preparing for the presentation I was blown away by how quickly Python is evolving (and I didn't even get to the new pattern matching features coming in 3.10). The slides are available here .
IMAPClient 2.2.0 is out! The most critical change in this release is a fix to avoid an exception when creating an IMAPClient instance under Python 3.9. imaplib (used by IMAPClient internally) now supports connection timeouts and this conflicted with IMAPClient's own timeout handling. Other highlights for this release: Python 3.8 and 3.9 are now officially supported Many performance…
use-package is an Emacs package which allows packages to be loaded declaratively. It's been around for ages and I've seen it used in other people's configurations, but I've only recently paid some real attention to it. I wish I'd learned how to use it sooner - it's really improved my Emacs config . Let's look at an example: (use-package magit :bind (("C-x g" .…
After a recent Christchurch Python meetup , I was asked to create a list of Python libraries and tools that I tend to gravitate towards when working on Python projects. The request was aimed at helping newer Pythonistas find a way through the massive Python ecosystem that exists today. This is my attempt at such a list. It's highly subjective, being coloured by my personal journey. I've…
I had the pleasure of giving a talk about Python virtual environments at this week's Christchurch Python meetup . It described the problem that virtualenvs solve, some gotchas and the tools people use to create and manage them. We also spent some time on some of the newer entrants in this space including pew , pipenv and poetry . The slides are available. Giving presentations is a great way…
I've just published a Jupyter Notebook I used to present an introduction to the excellent Python attrs package at the November 2018 Christchurch Python meetup . You can find it on Github .
influx-spout 2.1 has just been released and it includes a bunch of exciting new features. Here's the highlights... Downsampler The biggest addition is a new " downsampler " component which is useful for creating rolled up versions of metrics streams for long term archive. Generating rolled up versions of measurements in real-time is more straightforward than trying to create them from already…
IMAPClient 2.1.0 has just been released! Here's the main highlights: Python 3.7 is now officially suppported Testing against PyPy (version 2 and 3) Added support for the QUOTA extension Helper for locating special folders Document usage for using self-signed TLS certificates Document how to use the email package from the standard library to parse fetched emails Handling NIL values for…
As well as my main gig , I do some work with the excellent folks at Jump Trading . My main focus there so far has been finalising and open sourcing a project - implemented in Go - called influx-spout . influx-spout will primarily be of interest to you if: you use the InfluxDB time series database you have a lot of data going into InfluxDB you want flexibility in how incoming InfluxDB data is…
I recently had to write some NodeJS code which uses the AWS SDK to list all the objects in a S3 bucket which potentially contains many objects (currently over 80,000 in production). The S3 listObjects API will only return up to 1,000 keys at a time so you have to make multiple calls, setting the Marker field to page through all the keys. It turns out there's a lot of sub-optimal examples out…
I recently gave a talk at the Christchurch Python meetup which discussed the strengths and weaknesses of shell scripts, and why you might want to consider using Python instead of shell scripts. I went into some areas where Python is arguably worse than shell scripts, and then we dived into the excellent Plumbum package which nicely addresses most of those weaknesses. The slides for the talk are…
I'm very happy to announce that IMAPClient 2.0.0 is out. Many thanks to all the contributors who helped to shape what is one of the biggest IMAPClent releases to date. A major focus of this release was removing the dependency on backports.ssl and pyOpenSSL for TLS support. These dependencies were introduced in IMAPClient 1.0 to provide consistent TLS support, even for Python versions which…
IMAPClient 1.1.0 has just been released! Many thanks to the recent contributors and the project's other maintainers, Nicolas Le Manchet and Maxime Lorant. This release is full of great stuff because of them. Here's some highlights: search now supports nested criteria so that more complex criteria can be expressed. IMAPClient will add parentheses in the right place. See PLAIN…
This week I had the pleasure of giving a talk to Data Science masters students at the University of Canterbury (NZ). It was my attempt at giving some real-world advice from many years working as a technology geek. I think it went well and I hope the audience got something useful out of it. The slides are now available. I also managed to promote my current gig as well. Expect an article about what…
The Go compiler normally stops after it finds 10 errors, aborting with a too many errors message. For example: $ go build # sandbox/manyerrs ./errs.go:4: undefined: w ./errs.go:5: undefined: w ./errs.go:6: undefined: w ./errs.go:7: undefined: w ./errs.go:8: undefined: w ./errs.go:9: undefined: w ./errs.go:10: undefined: w ./errs.go:11: undefined: w…
I gave a talk at last night's Christchurch Python meetup about Python's relatively new asynchronous programming features. To be honest, I didn't know all that much about the topic and signed myself up for the talk to force myself to learn :) I used Jupyter Notebook for the presentation. Its ability to mix together text and interactive Python snippets works really well for this kind…
The code has been imported, tickets have migrated and the Bitbucket repository is now a redirect: the IMAPClient project is now using Git and Github. The Github link is: https://github.com/mjs/imapclient ( http://imapclient.freshfoo.com redirects to the new location) This change has been planned for a long time. I've noticed a that many contributors struggle with…
IMAPClient 1.0.2 is out! This is release comes with a few small fixes and tweaks, as well as a some documentation improvements. Specifically: There's now an explicit check that the pyOpenSSL version that IMAPClient is seeing is sufficient. This is to help with situations (typically on OS X) where the (old) system pyOpenSSL takes precedence over the version that IMAPClient needs. Use of…
Those paying attention may have noticed that this site has changed. Up until recently, freshfoo.com was generated using a combination of PyBlosxom and rest2web but it's now using the wonderful Nikola static site generator. The change was prompted by a desire to have more flexibility about freshfoo.com's hosting situation - moving a static site around is far easier than a dynamically…
The IMAPClient mailing list has been hosted at Librelist for a while. It worked OK initially but the site doesn't seem to be getting much love these days. People also tend to find the "send an email to subscribe" model confusing. After looking at a number of options, I've decided to shift the mailing list to Groups.io . It's a well run mailing list service which allows interaction…
IMAPClient 1.0.0 is finally here! This is a monster release, bursting with new features and fixes. Here's the highlights: Enhanced TLS support : The way that IMAPClient establishes TLS connections has been completely reworked. By default, IMAPClient will attempt certificate verification and certificate hostname checking, and will not use known-insecure TLS settings and protocols. In addition,…
Following on from last week's post about the upcoming IMAPClient 1.0 release, I'd like to draw attention to some significant, compatibility breaking changes related to the handling of search criteria. IMAPClient's methods that accept search criteria ( search , sort , thread , gmail_search ) have been changed to provide take criteria in a more straightforward and robust way. In…
IMAPClient 1.0 is really close to being done now and it's going to be one of the biggest releases in its history (thanks largely to the support of the good people at Nylas ). The headline feature of this release is the completely revamped TLS support. With 1.0, IMAPClient will perform certificate verification, certificate hostname checking, and will not use known-insecure TLS settings and…
I'm chuffed to announce that IMAPClient 0.13 is out! Here's what's new: Added support for the ID command (as per RFC2971). Many thanks to Eben Freeman from Nylas. Fixed exception with NIL address in envelope address list. Thomas Steinacher gets a big thank you for this one. Fixed a regression in the handling of NIL/None SEARCH responses. Thanks again to Thomas Steinacher.…
I'm very happy to announce that IMAPClient 0.12 is out! This is a big release. Some highlights: Unicode handling has been fixed. Some bad decisions were made during the Python 3 port (v0.10) and this release fixes that. Bytes are now returned in most places (instead of unicode strings). MODSEQ parts in SEARCH responses are now handled correctly. A crash has been fixed when MODSEQ queries…
I gave a presentation introducting IMAPClient at the monthly Christchurch Python meetup on Thursday. It included a brief introduction to the IMAP protocol, the motivation for creating the IMAPClient package, some examples of how it compares to using imaplib from the standard library, and some discussion of future plans. There also may have been some silly pictures. The talk seemed to be well…
I've been wanting to do this for a while: IMAPClient is now completely hosted on Bitbucket. The Trac instance is no more and all tickets have been migrated to the Bitbucket issue tracker. http://imapclient.freshfoo.com now redirects to the Bitbucket site. The primary motivation for this change is that it makes it possible for anyone to easily interact with IMAPClient's tickets…
I gave a talk last night at the monthly Christchurch Python meetup titled "Go For Pythonistas". It was an introduction to the Go programming language from the perspective of a Python developer. The talk went well, with plenty of questions and comments throughout. Thanks to all who attended for your interest! I'm not sure how useful they'll be on their own but the slides for the talk are…
This is a somewhat belated announcement that IMAPClient 0.11 is out (and has been for a little over a week now). Notable changes: Support for raw Gmail search API when connected to Gmail's IMAP ENVELOPE FETCH responses are now returned as Envelope instances. These objects are namedtuples providing convenient attribute and positional based access to envelope fields and the Date field is…
Now that they have officially launched I can happily announce that the good folks at Inbox are sponsoring the development of certain features and fixes for IMAPClient . Inbox have just released the initial version of their open source email sync engine which provides a clean REST API for dealing with email - hiding all the archaic intricacies of protocols such as IMAP and MIME. IMAPClient is used…
IMAPClient 0.10 has just been released. This is an important release because it's the first to support Python 3! Here's the highlights: Python 3.2 and 3.3 are now officially supported. This release also means that Python versions older than 2.6 are no longer supported. Be sure to see the NEWS.rst file for more information on this change. The HIGHESTMODSEQ item in SELECT responses is now…