Recent content on Stuart McColl
GitHub Universe takes place on November 8th and 9th this year, live and virtual. I’ve put together a list of ten sessions that I’m particularly looking forward to. Opening Keynote This year’s keynote will see GitHub CEO Thomas Dohmke and others look at what it means to be a software development platform in the age of AI. GitHub Copilot: the AI pair programmer for today and…
In this post, I’ll show you how to find deallocated Azure Virtual Machines using the Azure Resource Graph PowerShell module. Install the Azure Az PowerShell module We’re going to follow this guide on Microsoft Learn to install the Azure Az PowerShell module. I’m going to assume that you have the latest version of PowerShell installed. If you don’t, you can follow this guide…
I’m slowly working my way through my backlog of Microsoft Build 2023 session recordings, but wanted to take some time out to summarise my key takeaways from this year’s event. The common thread that ran throughout the event this year was artifical intelligence (AI), woven into sessions focused on what developers are building and how they’re building it, as well as sessions…
In this post, I’ll show you how to monitor system performance using PowerShell. I’ll introduce you to a few different cmdlets and some examples of how you might use them to monitor the performance of a system. Get-Process This cmdlet provides information about all of the running processes on a system, including their process ID, their CPU usages, and their memory usage. You might use…
In this post, I’ll show you how to create a Bicep file which declares an Azure key vault resource containing a single secret. Prerequisites I’ll be using Visual Studio Code as my editor, where I’ll be writing the .bicep file for this walkthrough. I’ll also be using the official Bicep extension. I’ll be using the Azure Az PowerShell module to handle deploying the…
In this post, I’ll show you how to validate Bicep files using GitHub Actions. What is GitHub Actions? GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform built into GitHub. It allows for the automation of build, test, and deployment pipelines. It can also be used for running workflows based on other repository events, like automatically responding to created…
In this post, I’ll show you how to find empty files using PowerShell. Find empty files in the current working directory The following command will retrieve empty files within the current working directory (e.g., it won’t recursively search within inner directories). It’ll return the full file path to any empty files that it finds. Get-ChildItem -File | Where-Object {…
In this post, I’ll show you how to find large files using PowerShell. Using the Get-ChildItem cmdlet Searching for files based on different properties can all be done using PowerShell’s built-in Get-ChildItem cmdlet. Find the largest files within the current working directory The following command will retrieve the ten largest files within the current working directory (e.g., it…
In this post, I’ll show you how to create a Bicep file which declares the resources required for an Application Insights instance, specifically one used to monitor a website. Prerequisites I’ll be using Visual Studio Code as my editor, where I’ll be writing the .bicep file for this walkthrough. I’ll also be using the official Bicep extension. I’ll be using the Azure…
In this post, I’ll show you how to find and remove empty Azure resource groups using the Azure Az PowerShell module. Install the Azure Az PowerShell module We’re going to follow this guide on Microsoft Learn to install the Azure Az PowerShell module. I’m going to assume that you have the latest version of PowerShell installed. If you don’t, you can follow this guide on…
In this post, I’ll show you how to retrieve Azure resources that do not have any associated tags. Install the Azure Az PowerShell module We’re going to follow this guide on Microsoft Learn to install the Azure Az PowerShell module. I’m going to assume that you have the latest version of PowerShell installed. If you don’t, you can follow this guide on Microsoft Learn. Run…
In this post, I’ll show you how to create a Bicep file which declares the resources required for an Azure Function App. Prerequisites I’ll be using Visual Studio Code as my editor, where I’ll be writing the .bicep file for this walkthrough. I’ll also be using the official Bicep extension. I’ll be using the Azure Az PowerShell module to handle deploying the resources…
In this post, I’ll show you how to retrieve Azure location names using PowerShell; specifically, using the Azure Az PowerShell module. Install the Azure Az PowerShell module We’re going to follow this guide on Microsoft Learn to install the Azure Az PowerShell module. I’m going to assume that you have the latest version of PowerShell installed. If you don’t, you can follow…
I’ve recently started experimenting with Bicep for deploying Azure resources declaratively. In this post, I’ll show you how to create a Bicep file which declares a simple Azure Budget with a monthly alert. Creating the .bicep file Create a new file, called main.bicep. The first thing we’ll declare in this file is the target scope of any resources that we create. In this case, the…
Something I’ve had to do on more than one occassion recently is copy the contents of one S3 bucket to another S3 bucket. There is no built-in way of doing this in the AWS Management Console, but it is possible using the AWS Command Line Interface (CLI). As an example, I’m going to create two new S3 buckets in my AWS account using Terraform, push some objects into my source bucket, and…
This year I created a small C# CLI application that scrapes this website and stores the results in Azure Blob Storage. It’s essentially a not-very-good version of the the Internet Archive’s Wayback Machine, but specific only to this website. The process happens in two parts. If I merge any pull requests on the website-archiver into main, then a GitHub Actions workflow pushes the latest…
Building and pushing Docker images to Azure Container Registry is simple with a small GitHub Actions workflow. This blog post assumes you’ve got a docker-compose.yml file in your GitHub repository. If you’re just using a Dockerfile, you’ll need to make a tiny amendment to the Build Docker image step in the push-to-acr.yml file described below. Microsoft Azure setup/configuration…
Another quick automation using PowerShell, this time to solve the problem of quickly adding a new GUID to the clipboard. The New-Guid cmdlet will quickly create a random globally unique identifier, with the below output. Guid ---- 8c312165-113b-4c30-91e9-e4e6edebcf0b This is great, but I’ve got a couple of extra needs; I don’t need the - characters, and I want the GUID to be added…
A simple automation need that I stumbled across recently was the ability to quickly create a .zip file from a static set of files and directories. Rather than run a long command in PowerShell each time, I bundled everything up into a quick function that I can run using New-StaticZipFile. The Compress-Archive cmdlet lets us create a compressed, or zipped, archive file. There are limitations (e.g.…
I keep some custom PowerShell modules stored locally on different machines, depending on the type of machine. For example, my work laptop has modules for setup and teardown of my development environment, which are modules that I don’t keep around on my personal desktop machine. Rather than have all of this live in a PowerShell profile which I do share between different machines, I keep a…
The Windows Package Manager client (also known as WinGet) celebrated it’s v1.0 release in May of 2021, after announcing at Microsoft Build 2020. At the time of writing, although the client is distributed within the App Installer package, it isn’t generally available unless you’re running a Windows 10 Insider build, or have signed up to the Preview flight ring. However, it’s…
If you’re using Windows Subsystem for Linux (WSL) in tandem with Visual Studio Code, you might be wondering how to change your default integrated terminal from PowerShell to WSL. Previously, you could set this in a single line in your settings.json file using terminal.integrated.shell.windows, but this has since been deprecated in a later version of Visual Studio Code. Add the following…
Using the shutdown command Setting a single, non-repeatable shutdown timer can be done using the shutdown command, which can be run in the Command Prompt or PowerShell. Let’s use PowerShell as an example. Search for ‘PowerShell’ in the start menu and open the app that is returned. Run the command shutdown -s -t TimeInSeconds where TimeInSeconds is replaced with the number of…
This is a quick blog post detailing what to do if you receive the push declined due to email privacy restrictions error when attempting to push to a GitHub repository. The error message tells us that your GitHub account has been configured to disallow the pushing of commits which might reveal a personal email address. This means that in order to push commits to this remote repository, you’ll…
Recently on the Digital Service Design Team at The National Lottery Heritage Fund we’ve been investigating Microsoft Azure blob storage as an option for storing files as part of a service we’re building. Before writing any code, we tested some of our assumptions about things like access and structure by calling the Microsoft Azure Storage REST API from Postman. This post documents a…
People on Twitter have recently discovered a GitHub secret. If you create a new GitHub repository with the same name as your profile name, you can add a README.md to your GitHub profile 1. How to create a GitHub profile README.md Create a new GitHub repository. You’ll see two mandatory options when creating your repository - the ‘Owner’ dropdown field and the ‘Repository…
In my last blog post I explained how to create list items with Microsoft’s legacy SharePoint REST API using Postman. This post builds on that work to attach a file to an existing list item. Assuming you’ve been following along, we have a SharePoint list with at least one item in it that we’ve created through a request sent from Postman to the Microsoft SharePoint REST API. The…
In my last blog post I explained how to authenticate requests to Microsoft’s legacy SharePoint REST API using Postman. This post builds on that work to store our access token as a Postman environment variable and to then send POST requests to the SharePoint REST API to create new items within an existing SharePoint list. If you were following along previously, the last thing we did was to…
Microsoft Graph API has largely superceded v1 of the SharePoint REST API for API-driven interaction with SharePoint online, but there are still a few things that you can’t do with it. For example, at the time of writing, it’s not possible to use Microsoft Graph API to add attachments to a list item, or retrieve attachments from an existing list item. The legacy SharePoint REST API does…
If you’ve tried to create a Microsoft Azure subscription using an existing Microsoft 365 (formerly Office 365) or Microsoft Power BI account, depending on how the account is setup you might have been redirected to this static page which displays the following error message: Your account belongs to a directory that cannot be associated with an Azure subscription. Please sign in with a…
Last week I wrote another AppleScript for automating a manual task I have to carry out each time I switch between my personal MacBook Pro and my work device. This time, I needed to write a script which would connect to my Bluetooth devices - a Magic Mouse, Magic Keyboard and my AirPods. Rather than accessing the System Preferences application, which I demonstrated in my last AppleScript, this…
AppleScript is a scripting language bundled within all versions of macOS, which allows for scripting of elements of the operating system and compatible applications. I’d never touched it before, but decided to try to use it to automate setting the resolutions of my external displays when switching between my personal and work MacBook Pros. Sometimes the resolutions stick when switching…
I moved hosting of my domains recently from tsoHost to Amazon Web Services, as uptime on tsoHost seems to have been slowly getting worse. Rather than using disparate providers for different services, I chose AWS to sit everything under one provider. Shifting to AWS itself was relatively easy - I’m now using Amazon WorkMail for my email service and Amazon Route 53 for DNS. The documentation…
The aggregator design pattern is a simple way of providing a single, unified service capable of surfacing data from multiple microservices, and a commonly used pattern when implementing a microservice-based architecture. Let’s imagine that we’ve been tasked with developing an internal API for our organisation - a general practice clinic. The API needs to consume data from three…
AWS Lambda lets us run code without provisioning or managing servers, paying only for the compute time of the running code. There’s no permanent infrastructure, and the server only has a life cycle of 40 milliseconds. AWS provides automatic horizontal scaling for Lambda applications, spinning up and down as many instances as are necessary. An open source Python library, Zappa gives us the…
How can we design APIs to be resilient when our networks necessarily aren’t? An API should be robust enough to handle failure scenarios including connectivity drops, timeouts between resultant calls, and more. If a client makes a request to our API and loses connection during the request, how can we ensure that a successive identical request doesn’t alter the state of the system in a…
PEP 557 in the recently-released Python 3.7 added data classes to the standard Python library. Data classes can be thought of as mutable data holders and are somewhat similar to named tuples, although named tuples are immutable. Data classes provide a lot of boilerplate code, saving time and effort on the part of the Python programmer, although it could be argued that this layer of abstraction…
I’ve recently written a small command line utility using Python 3.6 which will produce a CHANGELOG.md file from the commit differences between two different GitLab project branches. I’ve released this as an open source Python package and it’s available from PyPi here. Not intended to be a direct replacement for writing a manual changelog, the utility should be used as a draft…
Flask is a microframework for Python, based on Werkzeug and Jinja2. The core Flask framework is extremely lightweight, albeit infinitely extensible, and it’s simple for an experienced developer to pick up. I had been using Laravel a lot at the beginning of the year and Flask has been a breeze to work with in comparison. With it’s extensibility I haven’t been locked into working…
Kombu is an open-source messaging library available for Python which aims to make messaging as simple as possible. Kombu provides a high-level interface for the Advanced Message Queuing Protocol (AMQP), an open standard protocol for message orientation, queuing, routing, reliability, and security. The most popular implementation of AMQP is the RabbitMQ open-source messaging server. In the example…