Recent content on Techno Wizardry
It finally happened to me. I dared to log into my LinkedIn account and update my job status because I left my job. I dared to update LinkedIn while I was on vacation in Hawaii. When I tried to login, I found my account locked for security and they just needed a quick peek at my government ID to make sure I’m legit. Don’t worry, only LinkedIn and Persona have access they say. Ah yes,…
I’ve been modernizing my OVH-hosted dedicated servers running my various services moving from a CPU from 2015 to a CPU from 2019. While that’s still not that new, they’re budget servers and I only pay $30 USD a month for them. I managed to upgrade 2 out of 3 of them without any issues, making some improvements as I go, including enabling encryption at rest for all my data.…
Nix/NixOS is a declarative language for defining your entire operating system. I use it on my dedicated servers to be able to apply GitOps for the servers. I define my services in a Git repo, everything from what version of packages to use, to what services should be installed, and how they should be installed. Those servers run Kubernetes which is where most of my services live. Nix is a beast.…
In my home lab, I use cdk8s which builds on AWS CDK to define my Kubernetes resources infrastructure as code. A lot of people use Helm which uses YAML programming, which I wrote about why I didn’t like , but cdk8s allows me to write it in TypeScript. I can write reusable classes to reduce duplication, like an Ingress construct that handles all the configuration I need. If you want to read…
What’s in a Docker image? Nobody knows! Unless, you use an SBOM. What is an SBOM you say? Source Bill of Materials is a file containing all of the resources and tools that were used to build or are included in a software release artifact. SBOMs can be used to identify vulnerable software or even ensure compliance with software licenses. Imagine being able to block vulnerable software using a…
Kyverno is a handy policy engine for Kubernetes. For example, I use them to improve security by enabling user namespaces , or fix compatibility issues . I frequently use to validate or mutate resources that are created. However, the YAML format can be a bit tricky for some of my more complicated policies. LLMs do alright creating many policies, but can struggle with my more complicated policies…
As I switch from GitHub to Forgejo for my various project repositories, I find myself looking for a way to securely deploy new services into my Kubernetes cluster without having to keep updating Kubernetes auth tokens and storing them as a workflow secret. GitHub Actions and, as of Forgejo v15 , Forgejo Actions also supports providing temporary JWT tokens to workflow steps that securely state what…
In my Kubernetes, I sometimes try to run a pod on a specific worker node. Maybe one of them has a folder that I need or a specific hardware characteristic. Historically, I’ve used Pod spec.nodeName: srv5 However, when that node becomes unavailable, say because it’s run out of disk space and has DiskPressure on it, then Kubernetes will continually try to spin up thousands of pods on it.…
In my previous post , I showed how to provision a Kubernetes node in NixOS maintaining compatibility with RKE1 (Rancher Kubernetes Engine v1), but switching to the Kubernetes nixpkg . In this post, I’m going to show how to take an Ubuntu worker and replace it with a NixOS based worker without rebuilding the cluster. Replacing the OS using NixOS Anywhere For the first node, I was using a…
Everybody’s doing it. I guess I need to do AI, too. In my home, I have a few different tools that use generative AI and LLMs. I talk to my Home Assistant Voice Preview voice assistants which leverage a self-hosted Ollama running llama3.2 . I use Open WebUI , tried Tabby as an experimental coding assistant. I use DeepInfra for larger models that don’t fit on my own GPU. However, my…
Digital advertising is everywhere nowadays. However, they are actually a giant risk to privacy and now, safety. To be successful, digital advertising depends on showing you highly targeted advertisements, which ultimately incentivizes them to build up profiles about you via your browsing history, search queries, location, demographics, and even behavioral patterns. More data about you means they…
RKE1 (Rancher Kubernetes Engine 1) was Rancher’s first way of automatically deploying Kubernetes to a cluster. Think of it like minikube or on-prem EKS or K3s . Three years ago, it was marked as end of life (EoL) with the last release being July 2025. They have no migration guide and their strategy is just rebuild the cluster. I have 3 nodes a bunch of services running in Kubernetes. I…
Last year, I setup a Christmas lights show at my house. I started with some basic light sequences just to learn. I wrote a post on the basics. This year, I upped the ante and added more lights and starting making sequences linked to music. I have one light controller running Falcon Controller/FPP, a Kulp K8-B controller . How do I get sound out? I looked at options for getting sound out to a…
This is a continuation of my previous post where I talked about the challenges of using serverless/Function as a Service (FaaS) compute systems for ETL (Extract, Transform, Load) jobs. It sat in my drafts folder for a long time, so I just decided to publish it as is. I used to work at AWS, and predictably we used a lot of AWS cloud services. In many cases, when an engineer looks for a service…
Previously, if you wanted to subscribe to changes from this blog, you’d have to subscribe to the RSS feed , but as of today you can also subscribe to it in your preferred Fediverse client, like Mastodon . Note this is considered Beta quality. If you have any issues, let me know. What is the Fediverse ? It’s a protocol for federated (meaning many independently operated) social networks,…
I’ve been playing with Nix and NixOS a lot more lately. I installed NixOS on one of my servers , I installed the Nix CLI on my laptop, I tried to use Nix to build a Docker image, I use Nix flakes. This post was written from the perspective of a person new to Nix, but experienced with other computer languages. Thus, it’s probable that I might be doing something wrong or maybe…
For several years, I had 2x Unifi U6 Lite Access Points and they worked great. I had a special Wi-Fi network for my phones and laptops with a number of settings enabled but then I upgrade to the U7 Lite and immediately started having issues where my phone would disconnect. I got frustrated enough to break out my handy tool box to figure what was going wrong. As it turns out, proxy ARP was breaking…
When you run a container, the process IDs are namespaced and different in the container vs the host, the network stack is namespaced, the file system mounts are namespaced, but a process running as root in the container is running as root outside the container. This is risk because many privilege escalation vulnerabilities in Linux can be exploited because of this common user id. Linux user…
A few times in my Kubernetes clusters, I’ve encountered situations where some process consumes all the CPU or RAM which starves other services for critical services. For example, in one situation, Longhorn consumed all CPU and RAM and my pi-hole running on the same machine stopped being able to process DNS requests. Other issues have included having to shut down one of my worker nodes and…
In my previous post on Vault , I showed how Hashicorp’s Vault can be used to protect important passwords, static passwords that don’t change frequently. Vault can do much more than this and can even automatically create temporary accounts and rotate passwords for database users. Today, I’m using long-lived passwords that I generate once when I add a new service, I, along with…
I was recently working on an open source project ( tryfi/hass-tryfi - A Home Assistant integration for pulling data from my dog’s collar using the TryFi API and I found out that Git pushes can behave in a surprising way after I accidentally pushed a bunch of testing commits to the wrong branch. Background In my workspace, I had two different remotes. One that tracked my own testing repo and…
COE = Correction of Error My previous employer, Amazon, was a big proponent of doing blameless analysis of outages and figuring out what could be done to fix it. I recently had an outage on my servers and wanted to share what went wrong and the fix. Summary Starting Thursday until Friday, all TLS requests to a *.technowizardry.net domain would have failed due to a TLS certificate expiration error.…
Ever wondered how well your HVAC system is working in your home or condo? I did to an unhealthy degree. I want to know not just what’s the temperature, but how often is it running, what’s the supply and return temperatures, etc.? Let’s overengineer another project. To start, I’ve got an Ecobee thermostat and use Home Assistant to integrate with all my devices. Home…
I’ve had enough of Helm. I don’t know who thought string-based templating engines would be a good idea, but I have had one too many indention relate bugs. They’re a source of bug and a pain. Kubernetes YAML files just contain a ton of boiler-plate YAML configuration. Like how many times do I have to specify the labels? Its spec/template/spec for Deployment, but…
This blog is a static website compiled using Hugo . Up to this point, I built the website and packaged all of the assets into a Docker container with NGINX which was hosted on my dedicated server cluster. This worked well and was simple, but I have an upcoming project that I’ll be announcing soon that required dynamic content that nginx + pure static files wasn’t easily able to…
I posted a link to my blog on Slack and was greeted with HTML entities right in the website summary. I could see certain characters like the apostrophe ’ being encoded as ’ . Here’s how I fixed this problem. HTML Meta Tags Investigating Slack and social media sites use meta tags defined by the OpenGraph protocol to fetch information like the summary, publish dates, and images relevant…
I’ve been running my own mail server for well over ten years now. It’s pretty old, so it’s hard to make changes to it, but it’s running in Kubernetes. I was using a mixture of Postfix , OpenDKIM , OpenDMARC , and Amavis for spam filtering with SpamAssasin, but it wasn’t very good at catching spam. Instead, its time move to rspamd . It’s much newer and…
This Christmas season, I decided I wanted to play with programmable light strings and see if I could create an interesting light show on the front of my house. I stumbled across xlights , an open source light show sequencing program and got to work. The Hardware My light set-up needed a central light controller that would store the light sequences, send the light data to the strings of lights.…
I work at AWS, and predictably we use a lot of AWS cloud services. In many cases, when an engineer looks for a computer platform, they’ll often go directly to AWS Lambda because “it’s Serverless” with the justification that it’s simple and the best option no matter what and not want to explore alternatives. The FaaS (Functions as a Service) compute style is great for…
Today, I took a flight and tried to use the in-flight Wi-Fi, but I was unable to login to the the network. Nothing loaded or opened. I poked around in ip route and found two different routes that conflicted created by the Docker daemon. Looking at the following route, there’s two routes: 172.19.0.0/23 and 172.19.0.0/16 . These correspond to: 172.19.0.0 - 172.19.1.255 and 172.19.0.0 -…
I tried screen sharing in a video call on my Ubuntu 24.04 computer running the Snap Firefox install, but I could never get it to prompt to share a screen, thus it wouldn’t work. This post shows how I fixed that. Investigation My initial web searches came across this Stack Exchange question that recommended that users switch from Wayland to X11. While this did fix the problem, I’m not a…
For those not aware, Nix is an interesting new application (Nix) and operating System (NixOS) that provides a declarative environment definition and atomic operating system. Declarative means that instead of running apt-get install docker , you write down everything you want and it installs everything and removes everything you don’t want. You can use the same language to manage packages,…
In my Importing and cleaning my Mint transactions , I worked through loading, cleaning, and solving for transfers. However, Mint and other financial scraping tools are not authoritative and don’t expose everything that the bank itself will provide. For example, Mint and Monarch don’t have detailed enough stock transaction and position data to identify cost basis, tax lots, and…
I have several projects running in my Home Lab that now have to store and use sensitive secrets. In my Self-hosted finances series , I developed software to scrape my own bank statements (more on that coming soon.) In other projects, I store API keys to manage DNS or even my dedicated servers. These applications all run in Kubernetes, which does support Secrets , however, by default, they are not…
I use MQTT in my home lab to connect different Home Lab services like ESPHome , Home Assistant , Node Red , etc. It’s great because it’s a light-weight way to decouple these services, but by default there’s no security. I can’t prevent a sensor from manipulating another sensor’s data, I can’t prevent somebody who has network access from monitoring messages. In…
I recently got a Framework laptop and installed Ubuntu on it to give Linux for laptops a chance after using Windows and Mac for work for years. One thing I wanted was to be able to switch between light mode and dark mode automatically depending on the time of day. GNOME had a blue-light filter mode that could automatically turn on, but it didn’t appear to have a way to switch between light…
Previously, in my Self-hosted finances series , I cleaned and identified transfers in my Mint transactions for the purposes of of importing into Firefly-iii . In this post, I’m going to import the transactions into Firefly-iii. This part is comparatively easy vs the previous steps, however it’s only a one time import. A continue updating workflow is tricky and I’m working on some…
I own few domains and one of those domains is registered at GoDaddy. This is for historical reasons because this domain is on the .es TLD but my preferred registrar, PorkBun or CloudFlare, do not support this TLD. I kept it there mainly because I’ve had it for 10+ years and there were some new identify requirements that I didn’t want to deal with yet. I use external-dns as a tool to…
Over the past few months, as part of my self hosted finances series I’ve been working to extract all of my stock portfolio into some kind of self hosted database. I came across Ghostfolio , which is an open-source (with a paid hosted edition) tool for tracking stock portfolios. It was able to give me a portfolio view across multiple brokerages, automatically fetched stock prices, and gave…
Previously in Part 1 , I talked about how to clean-up the transaction data from Mint to remove duplicates and add any missing transactions. Solving for transfers The next phase is to solve for the transfer pairs. A transfer pair is defined with a matching credit and debit transaction on two different accounts. In Firefly, a transfer is treated separately than a credit/debit because it’s…
I was recently in the market for a new personal-use laptop and wanted to try out a Framework Laptop . I was intrigued by the idea of being able to replace any part that failed or even upgrade parts as I went. I also was frustrated with the direction that Windows 10 and Windows 11 was going. They seemed more interested in advertising, tracking, sending notifications to increase my engagement of…
Since Intuit announced that Mint was going away, I’ve spent several months investigating how to import my Mint data into Firefly-iii , an open source, self-hosted budgeting software. It seemed like a perfect fit. I would fully own the data and get to build whatever tooling I want on top. However, before we can get there, we need to have cleaned and accurate data from Mint. As it turns out,…
Birthing code is not always easy. Enough puns. Let’s talk about Java exceptions. No matter how hard you try, your code will likely encounter an error and throw an exception (if your language supports exceptions.) It could be anything from unexpected user input to an underlying service outage. An exception will be thrown and it’s important to do something useful with it. That…
In late 2023, Intuit announced that Mint was going to be shutting down and migrating everybody to Credit Karma. I could try out Credit Karma, but maybe it’s time to explore alternatives. Since that announcement came out, I launched a massive time sink to try and find a new option I liked. Intro Different people have different goals for a finance tracking app. My requirements are not so much…
I’ve been slowly reducing the amount of data shared with Google. I’ve been using Google Location History since 2013. I found it really useful just because I could figure out what restaurant I went to when I was traveling or any number of things. I found OwnTracks which was an open-source location history storage solution. It’s not nearly as polished as Google Maps where it…
Content-Security-Policy is a security feature ( MDN Web Docs ) in modern web browsers that restricts the kind of content that helps to protect against certain types of attacks, such as Cross-Site Scripting (XSS) attacks. Since my Home Assistant has significant access to my home network and is reasonably well-known, I wanted to take some steps to protect against malicious actors using XSS or other…
In a previous post , I noticed that all my Kubernetes services with type=LoadBalancer were exposing some internal services as NodePorts which meant that I might be exposing internal services to the Internet at high ports. I was running Kubernetes directly on my dedicated servers and not behind a load balancer. Kubernetes expected everybody to sit behind a LB which often times required a NodePort.…
I’ve worked on several different teams over the past 8 years I’ve worked at Amazon. Each one of them had on-call in which the engineers were on-call to keep the system running 24/7 for a week. If something broke at 2am, they’d get paged to fix it. Now, Amazon’s a big company. On-call varied quite a bit. Some teams had more ops load, others had barely any. I had my fair…
Quick, what is this diagram trying to show? An architecture diagram using AWS service icons to describe services I hope you know your AWS icons. There’s over 200 services and I have to guess frequently when playing the AWS Logo Quiz . While this diagram could easily add some descriptive labels to help, the icons assume developers can remember what the icon means. Some color-blind people may…
I’ve previously explored the world of home energy monitoring systems and in the past arrived at using the Brultech GreenEye Monitor for a project in a friend’s house. It had the advantage of being local out-of-the-box and had a wide range of compact CTs that made fitting the electronics in the breaker box a lot easier, but it had one flaw that made it not suitable for my condo. It had…