Tom Forbes' personal blog
After inadvertently finding that InfoSys leaked an AWS key on PyPi I wanted to know how many other live AWS keys may be present on Python package index. After scanning every release published to PyPi I found 57 valid access keys from organisations like: Amazon themselves 😅 Intel Stanford , Portland...
tl;dr : If you rely on the x-forwarded-for header with Cloudfront and have enabled Origin Shield, between October the 10th 2022 and November the 2nd 2022 the value of this header may have been incorrect for a percentage of requests. If your API relies on knowing the clients IP address in any way it ...
You can check out their website for a lot of buzwords , but it’s clear from all the stock photos that they take security Very Seriously Indeed ™️. However, from what I’ve found recently, it seems that Infosys use the following Comprehensive Management-Endorsed Proficiently Driven Cybersecurity Strat...
A few weeks ago I had the chance to delete 1 petabyte of data spread across 1 billion objects from S3. Well, actually 940 million, but close enough to the click-baitable 1 billion. I thought it would be interesting challenge to try and visualize the execution of these deletions and possibly gain som...
Update: Github have responded and said they will stop sending notifications about this CVE. Yesterday Github started notifying tens of thousands of people about a critical remote code execution vulnerability in a package named loguru. Their reviewed advisory is here. It references CVE-2022-0329 whic...
I’ve been very excited about RustPython since I saw the project being presented at FOSDEM 2019. Last week I finally got to contributing something to the project ! For context, RustPython benchmarks specific Python files against CPython, the “reference implementation” for Python (read: the only real ...
Cargo and crates.io are an amazing part of the Rust ecosystem and one of the things that makes Rust so pleasant to work with. However, it’s quite easy for your cargo dependencies to pile up without you noticing. For example you might blindly add the (rather awesome) reqwest crate, not realizing that...
What started with a simple image search for “Skippers hat” quickly turned into investigating a clever malware distribution network utilizing service workers, which is somehow linked to a group of hardcore vegan twitter bots. The hat at the entrance of the rabbit hole was innocent enough: However, wh...
When I was working on adding queryset.explain() to Django I got annoyed by how complex it was to set up a local Django environment with multiple databases and versions. The traditional way of handling this was to use django-box which utilizes Vagrant to spin up a VM and install different types of da...
I went through a phase that involved reinstalling MacOS way, way too often. At a certain point I got fed up with installing things over and over again and decided to version my dotfiles. And so https://github.com/orf/dotfiles was born. This reolved around a bootstrap.sh script that clones your dotfi...
I remember the first program I built fully on my own: a music downloader. There was a site that would basically index random music files found on Google and present you with a way to search them. I guess they would periodically search google for things like intitle:index.of mp3 (which you can still ...
Over two years ago I set out to try and improve Django’s autoreloader implementation. My PR was merged and the feature released in Django 2.2, and I thought I should at least mention it on my blog. The role of an autoreloader is conceptually pretty simple: When a developer changes a file your softwa...
I like Hugo a lot. It’s really fast and really simple. However it’s very much still evolving, and the theme I was using had not been updated in nearly a year. This meant that it broke with the new release of Hugo, and I was generally getting a bit sick of the design. Now I’m not a designer by any me...
tl;dr: check out https://applewatchfeatures.com/ , https://iosfeatures.com/ and https://macosfeatures.com/ I recently brought an Apple Watch. It’s a pretty fantastic product! One thing that quite annoyed me while evaluating if I should buy it is that features can vary quite drastically per country, ...
I’ve just released xcat 1.0 and it’s demonstration application after like 5 years of on-off development. Feels good! The genesis of xcat was when my boss, Sid, walked up to me out of the blue and asked if I wanted to go on an all expenses paid trip to Amsterdam. Who the hell wouldn’t say yes to that...
A few days ago I was invited to become a member of the Django software foundation due to my contributions to Django. Awesome! Now I get to hang about in the super-secret mailing list and discuss django-related illuminati business. Removal of core developers The hot news in Django-land is the proposa...
My work on adding bulk_update() to Django has been merged and will be released in Django 2.2! Like my filtered aggregates feature it relies heavily on the wonderfully versatile CASE statement to achieve some pretty good speedups for certain use-cases. The development documentation gives a pretty cle...
While working on any large-ish Django project you are bound to come across a slow query that’s perhaps missing an index or doing something else expensive. My workflow for diagnosing this was to get the query that is being executed ( str(queryset.query) ) and paste it into a database shell, prefixing...
Big Django projects often suffer from a few problems regarding database modelling and relations. Django provides incredibly easy to use tools to model your domain along with an awesomely powerful ORM to query on them. Often back office reporting software written in Django makes extensive use of the ...
Picture the scene. You’ve recently installed the awesome Lineage OS on your android device and are enjoying it so far. You install an update, it all goes smoothly, but upon restarting you are presented with this notification: What You press on the notification and you’re taken to the Lineage changel...
My recently merged PR for ticket #28398 adds very simple ‘did you mean’ suggestions to Django’s manage.py command, which is the primary way of interacting with Django from the terminal. So in Django 2.1 this is what you will expect to see if you misspell a management command:...
Many years ago, inspired by a blogging platform called Svbtle , I built a clone called Simple. It was pretty simple - just a minimal Flask app backed by a Sqlite database, but had some cool features. You could log in with Mozilla’s Persona (RIP), edit posts using an in-browser editor and drag ‘n’ dr...
I’m a full time nerd, even when I’m ordering pizza online I can’t stop myself from investigating how the websites I’m ordering from work. My latest investigation was Dominoes where I found a neat way to count the number of orders that they process throughout the day. This post is supposed to highlig...
I recently added syntax highlighting and support for CSS stylesheets to wordinserter , and the implementation was satisfying enough that I thought I would blog about it. Wordinserter is a library I maintain that lets you insert HTML documents/snippets into Word documents: It’s primary use case is wh...
American Fuzzy Lop is both a really cool tool for fuzzing programs and an adorable breed of bunny. In this post I’m going to show you how to get the the tool (rather than the rabbit) up and running and find some crashes in the cPython interpreter. Fuzzing? Explaining in detail what fuzzing is would ...
I often find myself creating one-off scripts to scrape data off websites for various reasons. My go-to approach for this is to hack something together with Requests and BeautifulSoup , but this was getting tiring. Enter Cyborg , my library that makes writing web scrapers quick and easy. Cyborg is an...
I’ve released a redesign of my HtmlToWord library, specifically it now supports Markdown and multiple different ways to interact with Word. It’s now also been renamed to WordInserter to reflect this. Originally HtmlToWord was designed to take HTML input, process it and then insert a representation o...
After discovering the flaw in Dell’s System Detect software I looked into other similar software for issues. This post details two issues I found with the HP Product Detection software and explores the protections HP put in place. I’m also going to explain how they could be easily bypassed to allow ...
I recently discovered a serious flaw with Dell System Detect that allowed an attacker to trigger the program to download and execute an arbitrary file without any user interaction. Below is a summary of the issue and the steps taken to bypass the protections Dell put in place. Timeline: The issue wa...
I’ve just about finished the next version of Simple , the markdown based blog that powers this site. When I first made Simple it was because I disliked WordPress, which seemed a bit too bloated. Then I saw Svbtle and I really liked the minimalist design (mostly the posting interface ) and decided to...
I just released XCat 0.7 , the companion tool to this paper. XCat is a command line tool to automate the exploitation of Blind XPath Injection Vulnerabilities , utilizing some pretty cool techniques. The most interesting technique is that xcat can automate out of band attacks to massively speed up e...
The coursework set for my Distributed Systems involves reading new items from RSS feeds (such as the BBC News feed or the UK traffic incident feed). To help me build the system I developed a simple service that serves up RSS feeds that are regularly automatically updated with nonsense items, and it ...
When I first came to University lots of people (like Rob Miles ) were trying to get undergraduates to start blogging. On the 6th of March 2012 I registered this domain and started blogging, getting myself added to the awesome Hull Compsci blogs syndicate. That was two years ago and a lot has changed...
I really like the Opera browser. A couple of months ago I got a bit tired of using Google Chrome, it was just a bit sluggish sometimes and I fancied a change. So I switched to Firefox, which I enjoyed using for a month or so until it too became irksome - it used a hell of a lot of memory and was als...
While working on a system for work I ran into a bug with Python’s lxml library and decided to fix it. I thought I would document how easy the process was, hopefully to encourage others to contribute to open source projects. Lxml is a “pythonic binding for the libxml2 and libxslt libraries” which put...
So I did a presentation on Information Security at University today. I think it went rather well, however I couldn’t show a couple of the demonstrations due to some SkyDrive files only being available online. That sucked because those were my best demonstrations, but overall I was happy. A few peopl...
I discovered an XSS flaw in a website a month ago and reported it to the owners. As a thank you they sent me a hat, a rather large american sized t-shirt and a pair of “DeFeet” socks (guaranteed to stay cooler and drier than any other brand). I didn’t expect them to ship something like that overseas...
Apache Thrift is pretty awesome - you can build Twisted bindings for your Thrift interface file that work fantastically. There is one thing that took me a while to figure out: I want to restrict clients connecting to the service to a specific set of IP addresses stored in a database. There were thre...
Tail-call optimization is a trick many languages and compilers use to avoid creating excess stack frames when dealing with recursive code like this:...
tl;dr The timetable system sucks, so I made one that works Getting your timetable sorted at Uni has never been fun. In years 1 and 2 of my study the department posted a timetable for each year showing all modules and students were expected to remove the classes they did not take, which while not the...
Regtransfers.co.uk is a website that allows you to purchase customized numberplates for your car or motorbike. They boast a large number of famous clients and short numberplates are often on sale for upwards of £20,000 (the plate ABC 4 is up for £30,000). While playing with their site I discovered a...
A week or so ago I was browsing /r/Python and I saw a link to a website called rise4fun.com , which is a Microsoft Research project that contains a lot of cool demos and tools that you can run in your browser. The demo I was linked to was a restricted Python shell that could be used to experiment wi...
Every once in a while I come across an application that is so comically insecure that I feel the urge to blog about it. The application in question is a .NET application to manage care homes and provide a Medical Administration Record for residents. Staff login to the app using a username and passwo...
Edit: Code is here on GitHub Calling functions in Python can be expensive. Consider this example: there are two statements that are being timed, the first one calls a function that returns an integer while the second one calls a function that returns the result of a second function call which return...
I recently brought a Samsung 840 Series Pro 256GB 2.5 inch SATA Solid State Drive and its easily the best PC hardware purchase I have ever made. Before I purchased it I was thinking about replacing my laptop as it was getting pretty sluggish, despite having decent specs even 2 years after I purchase...
A few days ago a colleague of mine asked me how you would pipe the standard output of a process into a browser. I hacked around for a few hours and came up with a websockets based solution (using Twisted and Autobahn.ws ) that you can see below ( Your browser needs to support WebSockets, sorry IE9 a...
My last post about the speed of Django’s templating language caused a bit of a stir and it was clear that people didn’t really have a clue how long the templates were taking to render in their applications. Enter Template-timings Template-timings is a panel for Django-debug-toolbar (which everyone s...
tl;dr Never trust user input. Note: Before I published this I contacted the owner of the site and he has since replaced it with a new and more secure version When I browse the net I often see the hallmarks of a security issue with websites that I visit, and little alarm bells go off in my head. I wa...
Edit 2: I made a Django debug toolbar panel that profiles your Django templates and all their components. You can find it here: https://pypi.python.org/pypi/django-debug-toolbar-template-timings Edit: It appears that Django does have some form of template caching. The graphs have been updated to inc...
I think I just solved a very first world problem - Where to find the nearest takeaway that sells fried chicken. When you are stumbling out of Spiders at 1am in the morning and craving some SFC goodness the last thing you want to do is wander the backstreets of hull in the cold looking for an open ta...