Inspired by this blog post and the talk that the author gave at KCD UK 2024 , I thought I’d write up how I do “Cloud Native” software development on macOS. The aforementioned post recommends using kind via Colima. There are very good reasons for doing this, such as that it can give you a multi-node environment in which to test things like Pod Disruption Budgets and affinity / anti-affinity rules.…
Something you might often do when deploying and testing your application on a local or a test development Kubernetes cluster is to hit the Ingress and test access. You’re almost certainly configuring TLS, and with that comes the question of what to do with x509 certificates. Typically you’d let the venerable cert-manager handle this for you, but what if you don’t want to battle with self-signed…
I recently picked up a second ThinkPad X1 Nano , a Gen 2 device, ostensibly to replace my older but much loved Gen 1 machine. It doesn’t really need replacing since it’s still a great little machine and easily my preferred laptop when travelling, but the deal I found for the Gen 2 was too good to pass up so here we are. Both this and the G1 run flawlessly under Linux; Everything is supported -…
Hot Snakes in Nottingham, back in 2004. As the tagline for this site and my online persona will attest to, the musical endeavours that Rick Froberg was involved in - notably Drive Like Jehu and Hot Snakes - had a profound impact on me. From a teenager hearing ‘Here Come the Rome Plows’ for the first time through to having my ears melted a couple of years back at what will sadly be the last time…
It’s been far too long since the last Sausage Cloud update, and a lot has happened that warrants an update and some news on what we’ve been up to. The platform itself is now much, much more resilient and performant than it was previously, and this is in no small part thanks to a few special folks and companies that have helped out on the hardware front. Infrastructure Networking Since Sausage…
Similar to my earlier entry on enabling GPUs with K3s or RKE2, this post builds on the process when you’re working with the NVIDIA GRID Operator and specifically RKE2 . Being an Operator, it introduces a custom controller into your cluster which encompasses the business logic required to automatically set up and provision GPU nodes within your cluster. In fact, I believe it’s your only option if…
Here’s some notes on getting GPUs working with K3s or RKE2. It’s pulled together from a few places to save folks the same trouble I found, i.e hunting through PRs and issues on GitHub to work out what needs to be configured in order for this to work. The first step is to get yourself a node with a NVIDIA GPU. In my case I’ve a 3080 configured using PCI passthrough to a VM running Ubuntu 20.04:…
New year, new blog post! This time it’s more magic with Cilium, which has a useful new feature that lets you specify a specific IP address in order to egress traffic from your cluster. This is really handy in very constrained environments, however the official guide suggests that you statically configure an IP address on the designated egress gateway node. This is a bit limiting, as if that node…
This post was partly prompted by the realisation that it’s nearly 2022 and I haven’t posted anything for the year… Anyway, here’s a snappily-titled post that demos a cool feature of Cilium - Cluster External Workloads - that lets you extend cluster networking to external clients, i.e other virtual machines, so that they can access resources hosted in Kubernetes. I’m a big fan of Cilium as aside…
Rancher 2.5 has ushered in a bunch of changes, and some of the functionality like backups and CIS scans have been moved out into their own Operators . It’s possible to make use of these on any Kubernetes cluster, not just one that’s been deployed and managed via Rancher, so this post details the steps necessary to deploy and run specifically the CIS Operator and view the results. First of all,…
This one’s a heads-up in case you’ve an OpenStack deployment that’s been around for a while and which hosts instances spawned prior to the Train release. Pets, if you will. You’re in for a bit of a shock if one of these instances is stopped and then started again - libvirt will refuse to create the domain, with a Nova backtrace along these lines: 2020-08-19 16:53:20.533 6 ERROR…
In an effort to inject some life back into this blog, I’m opting to do something relatively lazy and start posting semi-regularly with the odd link to anything that I’ve found of value or interest, but which don’t seem to have percolated their way up to the usual places. I’ve stolen the name (although it’s an obvious one) from Daring Fireball and it’s inspired to some degree by Scott Lowe’s…
In an effort to inject some life back into this blog, I’m opting to do something relatively lazy and start posting semi-regularly with the odd link to anything that I’ve found of value or interest, but which don’t seem to have percolated their way up to the usual places. I’ve stolen the name (although it’s an obvious one) from Daring Fireball and it’s inspired to some degree by Scott Lowe’s…
Update 12/2021: There’s a lot that’s changed in the last couple of years since this post was originally written, and it’s long overdue a bit of an update! Pretty much all of the ‘ancient-yet-spritely’ Blades have been replaced with much more modern servers in an effort to increase performance and also reduce power consumption. We’ve also upgraded to 10GbE networking throughout, and best of all we…
NB: As of kind 0.7.0 , the below is no longer necessary! 🎉 I’ve been using kind instead of Minikube recently for any Kubernetes-related hacking about - specifically for KUDO . Why? Well, on macOS at least, it’s quicker to launch and it’s also more efficient from a resource usage perspective - if you happen to have Docker running anyway that is. Remember that Docker for Mac actually runs Linux in…
NB: There’s a follow-up post here that goes into a bit more technical detail if that’s your thing… People that know me are probably aware of the fact that I’m something of a pack rat when it comes to hardware (and software). I hate throwing (or deleting) anything that still works and so might have some useful function, however unlikely - the box of ancient SCSI harddisks (9GB and 18GB!) along with…
In an effort to inject some life back into this blog, I’m opting to do something relatively lazy and start posting semi-regularly with the odd link to anything that I’ve found of value or interest. I’ve stolen the name (although it’s an obvious one) from Daring Fireball and it’s inspired to some degree by Scott Lowe’s “Technology Short Takes”. Angle Grinder . A CLI tool for manipulating logfiles.…
I’ve long had a bit of a thing for ‘classic’ ThinkPads. I owned a couple of X40s back in the day, and then a few years back I bought an X200 to lightly modify and run as an OpenBSD desktop. There are lots to like about these machines - decent keyboard, reasonable build quality, and by virtue of their popularity within open-source circles, excellent hardware support for operating systems other than…
Whoah, 2018?! Here’s some photos taken last weekend, whilst stood baking in the sunshine, at RAF Fairford for the Royal International Air Tattoo: There’s a bunch more over on Flickr here . Be grateful that the album only consists of 87 images - I actually took nearly 1600 photos over the course of two days. I also came to realise that my Nikon D700 is now nearly 10 years old! I’m probably due for…
“You don’t need a camera like that any more mate, we’ve all got smartphones!” The Hell Ride Blues made the trip over to Burnley last night to play a gig at The Turf Hotel, so here’s a few photos: And a couple more for good measure up on Flickr here .
A quick note on what to do if you find yourself in the unfortunate position of having to failover an LBaaS V1 pool from one agent to another. This isn’t supported via the API so you need to roll up your sleeves and dust off your favourite MySQL client. In this example, nn2 has failed and everything else has gracefully migrated away - I’m just stuck with some LBaaS pools that I need to move. In…
A quick one from my notes. If you’re blighted with logging spam from nova-compute along the lines of: Migration instance not found: Instance b4ff3ef3-3cb2-485a-b93c-f23d6dfc91ce could not be found. Then you’ve some failed instance migration metadata lingering in your Nova database that needs to be purged. Fortunately this is easy enough to fix, as long as you’re happy with manually deleting data…
I gave a presentation at the inaugural - and awesome - Docker Manchester meetup a few weeks back which detailed my “building Docker images with Puppet” workflow and why we’ve headed down that path. Since then it’s been refined a fair bit and so my original post is now a bit out of date. So, time for an update! Headline changes are: No longer building a ‘base’ Puppet image; A switch to Puppet 4 and…
Virtual machine live migrations are a crutch fact of life for pretty much anyone managing a sizeable estate. Working with KVM via OpenStack Nova is no exception; Upgrades, hardware failure, and general hypervisor maintenance often necessitate execution of nova live-migration to move an instance off onto another hypervisor elsewhere. This is all well and good, and in normal operation nova-scheduler…
Dartmoor’s a long way to go for weekend’s ‘glamping’ but we did it anyway. The third shot was taken with my iPhone 6S+. It’s unreal how far cameras in phones have come in the last few years. The last four photos are of the ‘Merrivale Rows’ .
The principles behind this post still stand, but there’s an updated workflow and tooling which solves a few of the problems mentioned below, described here . On and off for the last few weeks I’ve been trying to create a reasonably sane workflow that introduces Puppet - my favourite configuration management tool - to Docker - everyone’s favourite containerization (is that a word?) technology. I…
I’ve cobbled this post together from some older notes - there’s a few things in here that haven’t been a problem (for us ) since Juno, and with recent changes to Neutron i.e DVR and L3-HA some of it is lapsing into irrelevance anyway. But seeing as it came up recently on IRC I thought I’d put it together here just in case anyone else needs some help. Managing orphaned Neutron objects is something…
Update: Good news on the below. The issue with nova-manage db archive_deleted_rows has been fixed in Newton , and also backported to Mitaka . So if you’re running a fairly recent installation of Nova then you’ve a properly supported option for archiving this data prior to deletion. There’s also this blueprint which would handle purging the data entirely, hopefully slated for implementation in…
This isn’t so much of a ‘how-to’ (as it’s been documented with perfect clarity by Sébastien Han here ), it’s more of a warning when enabling the multi-backend functionality. tl;dr If you enable Cinder multi-backend, double-check the output of cinder service-list and be sure to update the host mappings for all your existing volumes. I came across a problem recently after enabling Cinder’s…
This past weekend saw Duxford and the IWM host an airshow dedicated to the 75th anniversary of the Battle of Britain. It was an incredible event with the sun shining almost all day long - here’s a few of the photos that I managed to take throughout the day: That last photo is mostly the closest I was able to get to capturing all 17 (!) Spitfires that took to the sky for the show’s finale. Seeing…
Chris and Catherine’s wedding was the perfect excuse to spend a few days in and around Chamonix. The weather couldn’t have been any better for the big day, but the rest of the time it was a bit hit-and-miss and meant that our trip up the Aiguille du Midi wasn’t ideal for sweeping vistas of the valley: Still, the ScanmaN came fully prepared with apple vodka, wine, and baguettes - so what was the…
It’s a fair old way to go for just a week’s stay, but the pain of long-distance travel was more than worthwhile for an opportunity to attend the first OpenStack Summit of 2015, and to soak up some of the sights and sounds that the fantastic city of Vancouver has to offer. Here’s a few photos I snapped during the bits of time we did have free away from the Summit itself: And a few more over on…
Having finally spent a bit of time with OpenStack’s Heat , I’ve started to see what I can do with automating infrastructure deployments and services by using it in conjunction with CoreOS . This post sort of builds on Scott Lowe’s introduction to CoreOS and Heat and does a few of the things he suggests, such as creating a dedicated network and deploying an arbitrary number of instances. It’s just…
From time-to-time user quotas in OpenStack become out of sync, i.e usage will show X number of instances in use and the reality, Y, is a different value altogether. Until now the way to fix this has been to manually update the nova database and amend the quota_usages table and then trigger a refresh by launching a new machine, i.e: update quota_usages set in_use = 0 where id = 890 and project_id =…
It’s surprisingly easy to find yourself with a large number of orphaned Neutron objects in your OpenStack installation, i.e resources left behind after a project or tenant has been deleted. This can be particularly problematic when it comes to routers and floating IP addresses as these can quickly eat into what precious few addresses you might have available, especially if they’re publically…
Here’s a quick guide for installing OpenStack’s CLI tools on OS X in a way that doesn’t make too much of a mess. First up we need to get a couple of additional tools installed that will ‘pollute’ your base install, but once we’re there we can easily create ‘virtual’ Python development environments and install isolated sets of packages on a per-project basis. This includes creating an ‘openstack’…
There’s a bit of gap in the current crop of OpenStack documentation, both official and unofficial, when it comes to doing any kind of end-to-end operational troubleshooting on the networking side of things. This series of posts is an attempt to rectify that and join a few concepts together in a way that other administrators will find useful (I hope!). Our installation mirrors the recommended…
Flickr album with a bunch more shots here .
A few shots from The Hell Ride Blues’ gig at Wangies (no, I’m not making the name of the place up…) last night as part of 2014’s Salford Music Festival . So good to see these guys in a context other than their basement - a lot more people deserve to see them play live.
Just over a year on from their last ‘underground’ gig and no less awesome. Someone should tell them that there’s a world outside of that basement though… A few more over here .
I’m sat here struggling to string together even a few words to say about our little visit to Sintra, its surrounding area, and Lisbon. Some places genuinely take you by surprise - in a good way - and that’s exactly what happened during our trip. It was an amazing week on every level. Even the excuse for going was perfect: the wedding of a couple of very awesome people indeed. But at least I took a…
I’m not a morning person by any stretch of the imagination. I’m also not really one for hiking and walking about or the ‘Great Outdoors’ in general, but an adventure like this - a hike up to the top of Pavey Ark in the Lake District might have just about changed that. Related: It’s my birthday tomorrow, just saying .
If you followed my previous guide (and haven’t done much with it since…), then you’ll have a basic install of ownCloud sitting in a Docker container as well as a corresponding base image. Although this is a good starting point, it’s far from ideal for a number of reasons - we haven’t configured SSL nor have we configured a slightly more efficient database backend (MySQL), for example. To get this…
Saturday night, at an undisclosed location in deepest darkest Manchester, we witnessed Hell Ride Death Punk Blues tear up a basement for a miniscule crowd of people. And what a treat. I did my best to grab some photos of the occassion, couple of my faves are below and be sure to check the rest on Flickr . Keep an ear (and an eye) out for these guys… They’ve got big plans, and rightfully so.
Docker is another piece of virtualisation-related technology that’s very much ‘in vogue’ at the minute. And for good reason - most of which will be immediately apparent and familiar if you’ve ever worked with FreeBSD’s Jails or Solaris Zones. It’s billed as ‘The Linux container engine’, with a primary goal of providing a framework for self-contained, easily redistributable and deployable images -…
More of the same over on Flickr .
Landscape photography isn’t usually my thing, but if there’s one way of changing that it’s spending some time in the Scottish Highlands. After nearly three (!) years in Edinburgh we finally took a trip out beyond our doorstep and tried to cram in as much sightseeing as possible over the course of a Bank Holiday weekend. The appearance of the sun was almost as incredible as the use of my camera.…
Sweated my arse off in Darmstadt’s Golden Krone to take a few shots of these guys playing at the ‘Nonstock’ warm-up. Click on the small ones below to enlarge, or check out the full set on my Flickr account over here .
My good friend Greg and his cohorts, known collectively as Married to the Sea , rolled into Darmstadt late yesterday to play an acoustic set at a locally (in)famous and unusual venue known as the Gute Stube. It offered an ideal opportunity to bust out the camera and test out some low-light shooting of which I’ve done very little with the D700. In short, I was blown away - not just by the music,…
Two or three years back I was lucky enough to purchase some rare records off a certain Mr. Joe McRedmond, who some of you might know as one of the members of Hoover and The Crownhate Ruin We exchanged a few emails and he offered up some unreleased TCR tracks which I downloaded, listened to, and that was that. He’s mighty nice, is Joe. Fast forward a couple of years to late last year, when I…