Recent content on blog.dataforce.org.uk
AI in coding, Hype or genuinely helpful? I’ve always leaned towards ‘hype’ but a recent dive into using it for some real-world tasks started to shift my perspective… slightly. I’ve long been generally skeptical of “AI” in all its forms. The idea behind it is fun, the technology is fun (if ethically problematic 1 ), but overall I’ve been rather…
What if I told you I’ve tracked every single night of sleep for the past 14 years using an app that’s basically been dead for over a decade? If you actually know me, then it probably comes as no surprise at all. Anyone who has spent any considerable amount of time chatting with me will know a few things about me: I don’t like change 1 . I like graphs and data. I do not always…
I’ve blogged before about Advent of Code and how I’ve been doing it each year since it began in 2015 and wrote a benchmarking tool for it - AoCBench . A few years ago when I first wrote AoCBench, there was limited formal guidelines from Eric with regards to repo content, specifically, the closest thing to “offical” guidelines for commiting inputs to repos was a couple of…
I have been a long-time user of proxmox for my virtualisation needs, starting back in the days of Proxmox 3 (prior to that I was running Ubuntu with OpenVZ rolled in by hand). Back then I only had a single server and resources were tight, so I deployed a lot of my workloads as OpenVZ containers. As time went on, proxmox switched to LXC, and I dutifully converted all my containers to LXC and kept…
I have recently added some iscsi-backed storage to my proxmox -based server environment, primarily as an off-server location to store backup data. For a multitude of reasons, such as the sensitive nature of the data, the fact that the physical storage lies outside of my control, and just good security hygiene - I wanted to ensure that the data is all encrypted at rest. I wanted to be able to use…
Once again, I have replaced the theme of this blog. Unlike the previous theme, this one is actually one I mostly ended up designing myself rather than just finding one that I mostly liked and running with it, and given that, I figured I’d talk a little bit about the thoughts behind it and how it came to be and what else I’ve done behind the scenes.
Since my previous posts about running docker-swarm with ceph, I’ve been using this fairly extensively in production and made some changes to the setup that follows on from the previous posts.
In $DayJob we make use of Dell S4048-ON Switches for 10G Top-of-Rack (ToR) switching and also sometimes 10G Aggregation/Core for smaller deployments. They’re fairly flexible devices with a high number of 10G ports, some 40Gs and they can do L3 ports and L2 ports. You can also run them either Stacked or in VLT mode for redundancy purposes. In addition these things use ONIE (Open Network…
In $DayJob we make fairly extensive use of MPLS ATOM Pseudowires (XConnects) between our various datacenter locations to enable services in different sites to talk to each other at layer2. The way I describe this to customers is that in essence these act as a “long cable” from Point-A to Point-B. The customer gets a cable at each side to connect to their kit, but in the middle of it…
This post is a followup to an earlier blog bost regarding setting up a docker-swarm cluster with ceph. I’ve been running this cluster for a while now quite happily however since setting it up, a new version of ceph has been released - nautilus - so now it’s time for some upgrades. Note: This post is out of date now. I would suggest looking at this post and using the docker-compose…
This all started with a comment I overheard at work from a colleague talking about a 2FA implementation on a service they were using. “It works fine on everything except Google Authenticator on iPhone.” … What? This comment alone immediately piqued my interest, I stopped what I was doing, turned round, and asked him to explain. He explained that a service he was using provided…
I’ve been wanting to play with Docker Swarm for a while now for hosting containers, and finally sat down this weekend to do it. Something that has always stopped me before now was that I wanted to have some kind of cross-site storage but I don’t have any kind of SAN storage available to me just standalone hosts. I’ve been able to work around this using ceph on the nodes. Note:…
For a few years now I’ve been enjoying Eric Wastl ’s Advent of Code . For those unaware, each year since 2015 Advent of Code provides a 2-part coding challenge every day from December 1st to December 25th. In previous years, Myself and Chris have been fairly informally trying to see who was able to produce the fastest code (Me in PHP, Chris in Python). In the final week of last year to…
I recently acquired a new server with 2 drives that I intended to use as RAID1 for a virtualisation host for various things. My hypervisor of choice is Proxmox (For a few reasons, Support for KVM and LXC primarily, but the fact it’s debian based is a nice bonus, and I really dislike the occasionally-braindead networking implementation from vmware which rules out ESXi) This particular server…
In my previous posts I discussed the history leading up to, and the eventual rewrite of my DNS hosting solution. So this post will (finally) talk briefly about how it all runs in production on MyDNSHost . Shortly before the whole rewrite I’d found myself playing around a bit with Docker for another project, so I decided early on that I was going to make use of Docker for the main bulk of the…
In my previous post about DNS Hosting I discussed the history leading up to when I decided I needed a better personal DNS hosting solution. I decided to code one myself to replace what I had been using previously. I decided there was a few things that were needed: Fully-Featured API I wanted full control over the zone data programmatically, everything should be possible via the API. The API should…
I run ubuntu on my servers, and since moving to Hugo, I wanted to make sure I was using the latest version available. The ubuntu repos currently contain hugo version 0.15 in Xenial, and 0.25.1 in artful (And the next version, bionic only contains 0.26). The latest version of hugo (as of today) is currently 0.32.2 - so the main repos are quite a bit out of date. So to work around this, I’ve…
For as long as I can remember I’ve hosted my own DNS. Originally this was via cpanel on a single server that I owned and then after a while I moved to a new server away from cpanel and moved to doing everything myself (web, email, dns) - hand-editing zone files and serving them via BIND. This was great for learning, and this worked well for a while but eventually I ended up with more…
For a while now I’ve been thinking of moving this blog to a statically generated site rather than using wordpress. There are a number of reasons for this: I can version-control the content rather than relying on wordpress database backups. It renders quicker I don’t actually use any of the wordpress features, so it’s just bloat, and a potential security hole. I’ve seen…
It did not help.
It’s been a while since I updated this. Not through a lack of wanting to, more a combination of things - lack of time, lack of anything worth writing, hating the old blog theme… So I’ve replaced the theme! Maybe that’ll help. It probably won’t.
I recently had the misfortune of having a server I am responsible for used as a target for DNS Amplification, and thought I’d share how I countered this. (Whilst this was effective for me, your mileage may vary, but if this actually helps someone then it’s worth posting about.) This particular server was the main recursor for the site that it was located at (And this was correctly…
Over the past few weekends I’ve been (slowly) working on moving my websites around a bit so that things are all in once place, and in the case of this blog, no longer hosted on my home ADSL connection. At the moment all non-existent pages on http://home.dataforce.org.uk/ , http://dataforce.org.uk/ and http://shanemcc.co.uk/ will now redirect to the equivalent link on…
A while ago I posted a python script that allowed automatically adding labels to GMail messages based on contact groups. Unfortunately, a side effect of this script was that Google occasionally would lock an account out for “suspicious activity”, and for this reason I stopped using the script. However recently I looked at Google Apps Scripts to see if this would allow me to recreate…
Update: This post still gets a lot of search traffic hits, but is now over a year old, and I no longer have a need to use Lync, so haven’t needed to keep this working. I believe that the Ubuntu repos now contain new enough versions of SIPE that the deb mentioned here shouldn’t be needed any more, but that the rest of the instructions should still be valid. Update 2: I need to use LYNC…
Compiling ip6tables on Endian Community Firewall (EFW) 2.4.0 Unfortunately the version of ip6tables available at the time of fedora core 3 doesn’t support the ‘state’ or ‘comment’ modules for use with firewall rules. So in order to get these, I decided to compile iptables 1.4.12.2 for Endian.
First post in over a year! Oops. For a while now, my home ADSL provider (EntaNET) has provided me with an IPv6 allocation, but I’ve never really used it (Its been on my to-do list for some time) primarily due to the fact that it is unsupported by Endian which I use for my home router/firewall. However the other day after being asked about IPv6 at my day job, I decided I wanted to get this…
<Obligatory happy new year post goes here>
So, a while back (just under 3 years ago) I obtained 2 of Hyundai’s W240D monitors. These monitors had (I believe) PVA panels and worked fine for most of their life so far. A while back they both developed a problem, as evident in the video below: So, as they were still under warranty I contacted Hyundai/RepairTech and arranged for these to be repaired. Hyundai sent the parts to RepairTech,…
If you’ve ever visited the hp.com forums you’ll know that any links in the post get enclosed by a call to “javascript:openExternal(’’)” in the href rather than doing it properly in onClick. Amongst other things, this breaks the ability to middle click to open links in new tabs. This finally annoyed me enough today and as a result, I now use the following…
I recently purchased a HP Compaq Mini 311c-1030SA with Nvidia ION and built in 3G, unfortunately the 3G card is a “UN2400” which isn’t supported right out of the box as it requires proprietary firmware. This post is mostly notes for myself on getting the UN2400 3G card inside it working enough to use. This post assumes that the netbook is running Ubuntu maverick (which is…
Today I decided to play with an old Huawei E220 I have lying around. After getting it setup and recognised in Linux by following the first 5 steps from http://ubuntuforums.org/showthread.php?p=3656717 After this, restarting udev ( restart udev ) and replugging the device makes it ready to use.
I have recently decided to switch this site to use wordpress rather than the custom code that was here before. This will allow me to edit/post to the site using my phone, which will probably allow me to update it more often. I am working on migrating everything from the old site to the new site and trying to find a theme I like. Hopefully I will have everything working soon. Should anything be…
I recently encountered a problem on a server that I manage where by the oidentd server didn’t seem to be working. Manual tests worked, but connecting to IRC Servers didn’t. I tried switching oidentd with ident2 and the same problem. After switching back, and a bit of debugging later it appeared that the problem was that the IRC Servers were expecting spaces in the ident reply, whereas…
Recently I setup gitweb on one of my servers to allow a web-based frontend to any git projects which the users of the server place in their ~/git/ directory. After playing about with it, I noticed that it allowed for placing a README.html file in the git config directory to allow extra info to be shown on the summary view, managed to get it to pull the README.html file from the actual repository…
Recently I acquired a T-Mobile G1 to replace my old T-Mobile MDA Vario 2 (HTC Hermes). All I can say about this phone is that it is quite awesome. I no longer need to run an exchange server to keep my contacts/calendar synced somewhere as the G1 syncs everything to Google Mail/Calendar. Its a really good phone and I recommend it to anyone who is thinking of getting a new phone, the integration…
As as I mentioned before I’ve been recently converting an old project to Java. This old project was an MDI application, and when creating the UI for the conversion, I found the default JDesktopPane to be rather crappy. Google revealed others thought the same, one of the results that turned up was: http://www.javaworld.com/javaworld/jw-05-2001/jw-0525-mdi.html So, I created DFDesktopPane…
NOTE: The information in this article has been superceeded by this one . As Noted by Chris recently on IRC, Google Mail lacks a feature in its ability to automatically label/filter messages - you can’t do it based on emails from people in a contact group, short of adding a filter with all their email address on it. At the time it was mentioned this didn’t affect me, however later when…
I was recently looking at converting an old application from VB6 to Java that used MD5 in its output files as hashes for validation. The first thing I did was to make a java class that read in the file and checked the hashes, I tried it on a few files and it worked fine, then I found a file that it failed on. Now, this app wrote all the files using the exact same function, so it seemed odd that 1…
Despite all the “This was amazing” “fantastic” reviews I seem to find for this everywhere, I found myself disappointed after watching it. The first 2 acts were funny and very rewatchable, there was gold bars that became gold liquid, the “Bad Horse” letters and phone calls, Captain Hammer and his “Hammer”, they made me laugh and alongside the humour…
For a while now (pretty much since I’ve been using Linux) I’ve been using yakuake , and I’ve been looking for something similar for OS X (at the moment I tend to ssh from my desktop to my OS X machine to do anything console like). I found visor which wraps terminal.app, but overall this is a poor replacement primarily for the lack of tab support (I use tabs a lot in yakuake, at…
Edit: This is now pretty much unneeded, the new version of VirtualBox seems to handle this all nicely on its own. As I mentioned in my last post, One of the useful advantages of the network boot setup is that I can use it to quickly install virtual machines. Now a few things: My Desktop is a lot more powerful than my server, so I run the virtual machines on it. I use virtualbox rather than vmware.…
So as you may or may not know from time to time I have the joy of fixing computers for various people. A lot of these fixes result in a reinstall of windows and away. This is a rather easy enough job, I have a KVM switch that I attach to the machine, pop a windows CD in (I used to have an unattended CD but don’t any more), answer a few questions and then occasionally switch the KVM over to…
Recently I finally finished adding support for AuthGate to be an OpenID Provider. This prompted Chris to develop Poidsy , which I have also implemented into AuthGate. This allows people to login to AuthGate (and thus also here, and other AuthGate powered sites) using OpenID instead of having to register with AuthGate itself.
On a daily basis, I get around 800 emails to my email accounts, of which most of it is spam. Now as good as thunderbird is at detecting spam, even it fails at a lot of the spam I receive, leaving me with around 100-200 spam per day that gets into my inbox. I’ve dealt with and accepted this for over a year now, before a discussion on IRC made me decide to do something about it. (When I say…
DMDirc 0.4 has now been released :)
I (Along with Chris ‘MD87’ Smith and Greg ‘Greboid’ Holmes have recently started working on DMDirc (again, only this time its in Java to help towards the original aim of the project to be a decent, truly cross platform IRC Client) Its coming along quite well :) You can track the progress, (and download the current release which is 0.1) at the Google Code Project or the…
With the release of windows vista, comes the start-of-the-end for Windows XP. with its EOL (End-Of-Life) date now set at January 30th 2008 (that’s less than a year away), people (by people I mean windows users) who are unable to upgrade to vista (due to Lack of computing power or so) or don’t want it (its crap, proprietary, riddled with DRM and probably bugs - Microsoft are already…
Due to a recent need in a project of mine for mySQL access from delphi/freepascal I have adapted the version of mysql.pas from http://www.fichtner.net/delphi/mysql.delphi.phtml to load both libmysql.dll (on svn) and libmysqlclient.so (Usually located in /usr/lib/mysql/). I also created a wrapper class for it (TSQL in SQL.pas) Downloads can be found here:…
Even the most complex captchas can be bypassed, if not tested thoroughly enough. Some time ago, I implemented a captcha on my comments form for news posts, to stop spam bots. My captcha is quite complicated, and sometimes generates images that even humans strugle on, yet for some reason I managed to get 10788 spam comments! I immediately tested my captcha - tried with no value, a wrong value, a…