Mac's Tech Blog
I'd like to thank ngrok for supporting my "art" (weird ranty tech blog posts). They paid me to write this post, but the opinions and stories here are my own. Many years ago, I was working in DevOps at Cisco. This was my first job out of college, so I had all the energy and wisdom of a sledgehammer. I fondly remember eating those Sysco burrito bowls from the Cisco cafeteria and getting flakes…
If I'm being honest, I feel like a lot of the technology we use day-to-day isn't super necessary if not actively harmful. But some technology is really helpful in the right context, like how moving a court hearing online can help people avoid being abducted by ICE. So we still need some technology, and the goal should be using technology that's accountable to and understandable by…
Recently, a ticket came across my desk asking for help with stopping people from using a free GPU service to mine cryptocurrencies. The tool they requested to do this was Falco , an eBPF-powered agent that runs in a Kubernetes cluster, watching for suspicious syscalls using a set of rules. To detect cryptomining, the default rules can detect outbound network connections to common miner pools by…
Recently, I completed a full rewrite of my first personal project , which was a Django project running on Heroku. Now it's a Deno project running on Deno Deploy. The experience has really gotten me thinking about the amount of pain we put up with to deploy simple stuff these days, especially with containers. Deno shows us that it doesn't have to be that way. I know what you're…
As we leave 2024, we're reckoning with both the fever pitch of AI hype and annual average air temperatures blowing past the critical threshold of +1.5 degrees Celsius which was set by the Paris Agreement 1 . With both of these happening simultaneously, there's been a lot of concern over the carbon footprint of tech in general and AI specifically: "Training a single AI model can emit as…
As someone who has built a career on Kubernetes, I'm always thinking about what "the next platform" is (for job security purposes). By "platform", I mean the kind that platform engineers like myself build for internal dev teams on which to run their applications. Kubernetes has been the star platform for quite a few years now (or maybe that's just my internet bubble), but will that last…
This post will make more sense if you first read Dear Sir, You Have Built a Compiler . Dear friend, I am afraid to inform you that you have built a Kubernetes. I know you wanted to "choose boring tech" to just run some containers. You said that "Kubernetes is overkill" and "it's just way too complex for a simple task" and yet, six months later, you have a pile of shell scripts that do not…
Recently at work I've learned about Cloudflare Tunnels , which has increased my interest in tunneling technologies in general. Tunneling is a generic networking term, but in the web-dominated world it usually refers to software that you can run on any computer with outbound network access to serve a website. The tunneling software has a another component running in the cloud that configures…
If you're reading this blog, there's a high chance that you are one of the 1-in-5 workers who has a non-compete clause hiding in your employment contract. These are clauses that prevent workers from taking other jobs or starting other businesses that "compete" with the employer after leaving the job. I originally thought they were unique to highly-paid tech jobs, but actually they are…
Lately I've been obsessing over this ProPublica article which talks about Microsoft's involvement in the SolarWinds hacks, specifically due to a design weakness in Microsoft's Active Directory Federation Services (ADFS) . It's a great article, highly recommended. Anyway, the wiki page for ADFS says it first appeared around 2003, which made me wonder: what did the cybersecurity…
Back in January SourceHut and Codeberg both experienced a distributed denial of service (DDoS) attack . That attack made me worried about the future of the small, independent Internet: services like SourceHut and Codeberg which do not use the oligopoly of large hosting providers. DDoS attacks are where an attacker controls a bunch of computers (usually a botnet of hacked devices with residential…
Last year I left a comment on HN about Docker Compose that seemed to get some traction. I was complaining about how both Docker Compose and Kubernetes are leaky abstractions over infrastructure. My general stance is that the Kubernetes API, despite its vast complexity, allows you to describe like 99.9% of modern applications. All attempts to simplify that API surface (such as Docker Compose's…
I wanted to write this because I don't hear enough real people discouraging the use of Web Application Firewalls (WAFs). Probably because the search results for "Web Application Firewall" are all written by WAF vendors. Anyone reading just that could conclude that WAFs are a good idea. I'm here to offer another perspective, after having suffered through using a WAF for two years. Web…
I recently picked up a few books to read, and Security Chaos Engineering engrossed me so significantly that it 1) made me read the whole thing and 2) made me want to write about it too! If you just want a taste of the book, check out the primary author Kelly Shortridge's blog . Her unique writing style is quite fun to read. My interpretation of the book was that it is the InfoSec world's…
This is a story about how in 2021, I discovered a vulnerability affecting an unnamed billion-dollar company and disclosed it to them, earning my largest bug bounty ever! Accidental Discovery At first, I was just doing some work on kubecost , a Kubernetes tool which estimates costs for running pods. The first thing that caught my security eye was that by default, when you install kubecost, it…
Back in 2020, A Russian state-sponsored group got into SolarWinds' build system and inserted command and control (c2) code into a routine software update for a network monitoring tool called Orion ( wiki link ). It was all over the news, and for good reason given the extent of the breach (into particularly sensitive parts of the US government) and the lengthy recovery process which will…
Unlike many other industries like doctors, real estate agents, lawyers, social workers, etc. the tech world doesn't have to abide by any particular code of ethics. We can argue all we want about if such a thing should be required or what should be in it, but I think clearly something is better than the nothing we have today. In most of the US at least, those professions I mentioned have…
If you didn't hear, CircleCI recently released their report on a December 2022 security incident . What stood out to me is that CircleCI seems to be doing all the "normal" security things: SSO with 2FA, endpoint protection, auditing/logging, encryption at rest, etc. But they still got breached. I think this speaks to the unfortunate state of the modern-day security industry. It's a…
Take a second to think through all the "stuff" you have to personally maintain at your job. Here are a few ideas: Physical servers or virtual machines Internal libraries A handful of microservices Some test cases you wrote CI/CD stuff Helper scripts Open-source repos Security/legal compliance Software licenses All of the tools on your computer (IDE+extensions, shell+extensions, CLI…
Securing Kubernetes is complex, so there are quite a few guides out there: CIS Benchmarks for Kubernetes (free account login required) NSA/CISA Kubernetes Hardening Guidance Kubernetes Pod Security Standards Various , other , sources Unfortunately, following all those guides and patching every CVE still might not be enough. There are some security practices which kinda don't fit into…
In 2016 , Kubernetes v1.3 was released which included a new API type: PodSecurityPolicies (PSPs). The original design proposal had the lofty goal of allowing cluster admins to restrict various Linux privileges to some Pods while still allowing other Pods to use them. PSPs filled an important security hole where having the ability to run pods could let you bypass every other security control in the…
It's no secret that Kubernetes Secrets are just base64-encoded strings stored in etcd alongside the rest of the cluster's state. Ever since the introduction of Secrets in 2015 , armchair security experts have been scoffing at this decision and seeking alternatives. I think those people are missing the point. The design of the Secrets API dates back to before Kubernetes v0.12. In a thread…
Since cyber attacks , a building collapse , and an infrastructure bill have been in the headlines recently, I think the time has come for us to re-assess our relationship with maintenance . While I work in the "tech" field, I don't align myself with inventors, entrepreneurs, or green-field software engineers. I align myself with the whole spectrum of "maintainers", like civil engineers, power…
To those of you who give advice on which languages, methodologies, architectures, or whatever else to use, I want to start by saying thank you. Your advice is vital to the growth of millions of people like me who are early in their careers. But I've noticed a pattern with software engineering advice that leads to over-engineering and unnecessary arguments. So I have a request: Please make…