We need to make our work lives safer. rmrf is intended to demonstrate how we can build safer tools for operating complex systems and make operators' daily lives more predictable and less stressful. tldr; I’ve created a tool called rmrf , a safer version of the well-known and extremely useful command rm -rf . Howver, rm -rf is a tool that with extremeley sharp edges (ask me, I know!) and it can be…
Identify AI-assisted code artefacts that require fixing. tldr; I am trying out new ways of testing AI-assisted code generation. LLMs will introduce all kinds of strange elements into the code or simply refuse to implement it. One approach in terms of detection is to use another LLM to identify any TODOs or other incomplete or unimplemented features in the code, as well as any weird artefacts left…
Model Context Protocol is new, so we need to build all the security tooling and process and knowledge around it. Just like when we invented TCP/IP we needed to invent firewalls and all that jazz. tldr; Model Context Protocol is new, so we need to build all the security tooling and process and knowledge around it, and thus there is a lot of new thingamabobs out there on the capital-I Internet, from…
Putting a form on the Internet is still crazy hard. There's quite a few steps to take to make it reasonably secure. tldr; If you want a “contact us” form on your site, it’s best just to pay for a form provider. On the other hand, there are some great free tier services that provide a lot of the functionality you need to secure a form, though it will be a fair bit of work. The fact that forms are…
We all have to get used to a new way of programming, and a new attack vectors that don't look like code. But they are. But they don't look like code. It's Halloween every day now. tldr; AI agents and applications need access to tools. But how do they decide which tools to use and how to use them? Natural language, that's how. For example, the use of docstring descriptions of a Python function (aka…
What do these lazy AI systems do? Not a whole heck of a lot without tools and resources. Use the model context protocol to give them a virtual jolt of caffeine! ⚠️ Please note that MCP is moving historically fast! What I wrote here today, is likely not what you will find on the MCP website tomorrow. I will update this post as needed, but please check the MCP website for the most current…
Make your Kubernetes networking life easier with the Tailscale Kubernetes operator. tl;dr Modern Kubernetes networking involves multiple complex layers: physical networks, CNI plugins like Calico (handling BGP meshes, VXLAN, IPIP tunnels, and NAT), service meshes, and external access concerns. While the post dives into Calico’s implementation details in a homelab setup—showing how it manages IP…
tldr; Malicious code is everywhere, but it's dangerous and hard to find. Here are some of the ways one might find it, or create it. ⚠️ Disclaimer: Malicious code (MC) analysis is important for cybersecurity, but it requires careful handling. We tend to keep it hidden, but it's out there (unfortunately, we also tend to find it when we least expect it). When performing malicious code analysis,…
🐍 tldr; There are many methods to distribute and install Python programs. Here are some of the ways I've tinkered with recently. Recommendation: in most cases, use pipx. 🔧 ⚠️ Disclaimer: I'm not a Python expert, I've just been writing more Python lately and trying to figure out how best to distribute it. There are several fascinating and useful ways to distribute Python programs, and I've tried…
tldr; Tailscale allows you to create your own private, secure networks, like the LANs of yesteryear, and now they can have "exit nodes" that are Mullvad VPN servers. I’ve been a Tailscale user for a while now, though my use waned as I struggled to deal with using a VPN as an exit node. For real, that was my problem. I really struggled with the exit nodes and VPNs. For a while I had a tailscale…
tl;dr: falcoctl is an attempt to make it easier to distribute and upgrade Falco artifacts, such as rules and plugins. What is Falco? Falco is a cloud native security tool that provides runtime security across hosts, containers, Kubernetes, and cloud environments. It leverages custom rules on Linux kernel events and other data sources through plugins, enriching event data with contextual metadata…
I recently started building a small Python application, less than 1000 lines of code. However, it’s been a real challenge to get it to the point where it can be deployed and run in a production environment. I’ve done this before, but it’s always difficult, and each time I’m reminded of how much effort it takes, in part because I always have to start anew. As a part-time developer…which of these…
In this post I’ll show you how to install and setup Incus on a physical host running Ubuntu 24.04. I’ll setup a storage pool and a bridge network, then launch a VM. Once this is all done, I’ll have a nice homelab server that can spin up many virtual machines and do it quickly, putting them on the right storage pool, on a separate network. What is Incus? Incus is a next-generation system container,…
Incus Virtual machines remain the main building block of pretty much all infrastructure. We tend to forget about the technology and just how entrenched it is in our daily technical lives. Ok, that’s a pretty heavy statement for a blog post, but I do think we forget about virtual machines and just how valuable and secure the technology is–most public cloud services are loss leaders for the VM part…
tldr; tldr; I built an insecure web application (on purpose) for testing LLMs and here it is: https://github.com/ccollicutt/insecure-nextjs-guestbook . Dealing with Technical Debt using GenAI Is cybersecurity largely a technical issue? An engineering issue? It’s difficult to say. Certainly human psychology plays a big part of it, but, then again, we’re building (insecure) software things and…
I’ve been working with Kubernetes for a long time. Too long, actually. So long, in fact, that I don’t really use it much anymore. Kubernetes has won in terms of being the default way to deploy modern applications. At this point, it’s kind of boring, which is great! We want boring infrastructure. Boring works. If you’re writing a new application today, the target is going to be a container, and…
I’m a fan of the recent wave of AI–specifically generative AI, or GenAI for short. I think of GenAI, these large language models, as a kind of compression. They take huge amounts of text–programming code, for example–and they are able to regurgitate it. So we take terrabytes of code, compress/feed it into an LLM that ends up being only a few gigabytes in size, and we can then talk to that LLM in…
There are a few ways we can customise a Large Language Model (LLM), and one of those ways is to fine-tune it. But why fine-tune an LLM? Large language models (LLMs) like GPT-3 and Llama have shown immense promise for natural language generation. With sufficient data and compute, these models can produce remarkably human-like text. However, off-the-shelf LLMs still have limitations. They may…
First, we start up ollama. $ ollama serve OK, it’s running. $ curl localhost:11434 && echo Ollama is running Now, let’s run Meta’s Llama 3:8b. NOTE: This is only the 8b model, the smallest one so far, AFAIK. $ ollama run llama3:8b pulling manifest pulling 00e1317cbf74... 100% ▕█████████████████████████████████████████▏ 4.7 GB pulling 4fa551d4f938... 100% ▕█████████████████████████████████████████▏…
I’m working on running LocalAI. But I feel like running that out of Docker. So how to use a GPU with Docker (on Linux). First, need the nvidia-docker2 driver. Otherwise you get an error like this: docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]. So install that. sudo apt install nvidia-docker2 I had a fun thing to fix in that I had added some things…
Well, last week I picked up the 3090 GPU. This week I need to try to use it. That is not an easy feat because “drivers.” My good old workstation is on Ubuntu 20.04. I should probably upgrade. I should probably not use this machine for AI work. But, I am. Currently I’m using the nvidia-driver-470 that I’ve had for a while, as though it’s some sort of cherished antique that I’ll hand down to my…
There are a couple of ways to think about this post: Option 1 (boring): I bought a video card and installed it in my computer. Option 2 (cheesy cliché cyberpunk; more fun): In the gray low-rent business suburbs on the edge of the city, where the air hums with the buzz of a thousand illicit transactions, I found myself trudging through a seedy strip mall, its flickering signs casting long shadows…
I will tell you a secret—no, a story. Say, at some point, I had a Kubernetes webhook admission controller that I wrote and deployed, and then the TLS certificate was automatically (nice!) renewed by cert-manager, but the pod wasn’t restarted, so it still had the old certificate, and now all Kubernetes deployments failed. That is indeed a story, perhaps a sad one. I had this shiny new cert, but no…
Security is a prime concern when deploying applications in a Kubernetes cluster. One of the security aspects in Kubernetes is controlling who can run what and as whom within a Pod. Kubernetes provides two important fields in the Security Context to achieve this: runAsNonRoot and runAsUser. While they might seem similar at first glance, they serve different purposes. This blog post aims to…
You’ve just deployed your Python Flask app on Kubernetes. You’re using Gunicorn as your WSGI server, and you’re trying to test how the container would behave if the app crashed. But wait! You find out that the container never crashes. Why not? Oh, Gunicorn is being helpful–it keeps restarting the application. OK, maybe this isn’t “you” it’s “me”. I was trying to build a demo app that showed crash…
Kubernetes is a framework. We don’t usually describe it as a framework, but it is. IMHO, it’s a library we can use to deploy applications and imprint our organization’s policies and requirements on top of. That’s what makes it valuable, not the fact that it can create a container. Because it’s a basic framework, a set of lego blocks, it’s not designed to be secure “out of the box.” We’ve got to…
In my opinion, the default level of security in Kubernetes is not enough. There’s some work that needs to be done to bring it up to some, perhaps arbitrary, level of security. This post is part of an exploration of that area. We used to have something called Pod Security Policies that we could use to increase the level of security, like reduce some exposure, but that model was deprecated . Now we…
tl;dr I built an app called chain-link that will create a “chain” of apps in Kubernetes of an arbitrary length. It’s written in Python. While the point of this all is the app, what I learned most about what writing the CLI portion: the chain-link-cli . There’s actually way more code there (for better or worse) to deploy and manage the app than there is in the app itself. What is it? I wanted to do…
Creating new deployments on a Friday is NOT allowed. Would you create a Kubernetes Admission Controller to block deployments on a Friday and use it in production? No. But you could, create one, say, as an example admission controller. So that is precisely what I have done, created a very, very (very) simple admission controller, written in Go, that blocks NEW Kubernetes deployments on a Friday. I…
The racoons are injecting the CA bundle, of course. I’ve been working on a simple validating webhook for Kubernetes. More on that later. However, one of the things that you need to provide when you create the Kubernetes manifest for a validating webhook is the CA bundle that the kube-api can use to validate the webhook. But…where does that come from? How do we get it into the manifest? Here’s the…
I work a lot with Kubernetes. So I need to have Kubernetes clusters. The way that I have usually been building them is with the Killer.sh training courses scripts, which can be found here: https://github.com/killer-sh/cks-course-environment/tree/master/cluster-setup/latest I decided to take that script and update it and change it around a bit for my liking. The changes I’ve made can be found here:…
I recently set up a second AWS account just to use for testing. I have a primary account, but I want one that I can easily wipe out absolutely everything in, specifically using AWS Nuke. What is AWS Nuke? AWS Nuke is a CLI applicaiton that can wipe out everything in an AWS account, if you want it to. What does it do? It removes everything from your AWS account. And I quote: Remove all resources…
I wanted to try out using Kubernetes from Jenkins, and that is what this post is about. I have a Jenkins instance running on a host, specifically it is NOT running in Kubernetes. But I want that instance of Jenkins to talk to a Kubernetes cluster and use it as a “cloud”, where I’m using the term “cloud” in Jenkins parlance. tl;dr Install the Jenkins Kubernetes plugin Set up the Kubernetes…
tl;dr I recently starting working in security again, at a great company called Sysdig I think the security industry has really been improving as of late…more work to be done, but I see progress I need to learn what a lot of security related acronyms mean Here’s a good podcast on security acronyms Categorization and definitions are ongoing Background I started my career in security. That was a long…
I’ve got (what I think) is a fairly substantial, though definitely aging, homelab. I thought I’d write a bit of a post on what it consists of. Hardware Current setup: 3x ESXI hosts - each are a 1U Supermicro server with 192GB memory on a X9DRI-LN4F+ motherboard 1x Network storage server - A 2U Supermicro, 64GB of memory, X9DRI-LN4F+ motherboard Intel Xeon CPU E5-2650 2.00GHz CPUs (old!) 1x…
In the last post I looked at creating secure software supply chains with the Tanzu Application Platform (TAP). In that post I used a default supply chain. But what if we wanted to create our own, custom supply chain instead of using the “out of the box” examples provided with the platform? Quickly…What is the Tanzu Application Platform? TAP is: VMware Tanzu Application Platform is a modular,…
If you are a company that makes software, then you have a software supply chain, whether you want one or not. Building software is challenging, even without thinking about where all the underlying dependencies and other software comes from, and what’s in it, never mind cataloguing and checksumming it all, and being able to replace it within a few hours. This is all hard work, work that many…
Tanzu Kubernetes Grid (TKG) is VMware’s distribution of Kubernetes that can be deployed into vSphere and public clouds like Azure and AWS. Sometimes customers prefer that the deployment and management of TKG is done “offline” without needing to obtain any artifacts of the deployment from Internet hosted resources, that the deployment is self-contained. This is a fairly common requirement,…
My current favorite VMware document is this: Running Tanzu Application Platform Locally on Your Laptop If you run through the above how-to document, which is affectionately called TAPonLAP, at the end you will have a functioning Tanzu Application Platform (TAP) profile-defined environment to use, and it’s all running locally on your personal workstation. What is TAP? TAP is: …a modular,…
Previously… In an earlier post I deployed a simple demo container image into Kubernetes via knative, and knative itself was installed as part of VMware Tanzu’s Cloud Native Runtimes, which is also Part of the Tanzu Application Platform. If that sounds like a lot, that’s OK, it is a lot. Tanzu Application Platform (provides ->) Cloud Native Runtimes (provides ->) knative All of this is using the…
tl;dr The point of this post is to show that knative is part of The Tanzu Application Platform , AKA TAP, and one can use knative outside of TAP, directly with the kn CLI, if desired. (Though, in the real world you would probably use the rest of TAP as well, but it’s modular so you don’t have to.) In this post TAP has been deployed into a GKE cluster (yep, a GKE cluster). TAP includes knative, so…
Azure has functions. Azure as a way to manage secrets called Key Vault. How do these work together? If you create a function and you want to access a Key Vault secret, clearly it has to authenticate to the Key Vault service…but how? Managed identity is the answer. But what is “managed identity”? A managed identity from Azure Active Directory (Azure AD) allows your app to easily access other Azure…
What Do I Want to Do? I’ve not used Azure Functions before, so I’m going to run through a quick start to deploy an example nodejs function. What I have/want to do: Already have az installed and an Azure account to work with Running from a Linux workstation Don’t want to use VSCode integration currently, prefer to use CLI for now Deploy a NodeJS 16 “hello world” function manually Installing Docs…
Secrets. I need to manage them. As part of my exploration of developer experience I plan on having many micro-services running on different platforms. However, I don’t want to have to manage the secrets across all of them individually…that would be a nightmare, never mind thinking about various environments (dev, test, prod…). For example, I’m using Basic Authentication as a simple API key and API…
Developer Experience Working at VMware in the Tanzu group I’m always focussed on developer experience (DX). Often people think of VMware as an infrastructure only company, but we’re not. Tanzu is heavily focussed on developers. I’d say 10-20% of what we do is infrastructure related (Kubernetes, Cloud Foundry) and the other 80-90% is related to devops, security, developers, and software architects…
In this post we’ll deploy the Tanzu Build Service (TBS) onto a vSphere with Tanzu Kubernetes workload cluster. Requirements vSphere with Tanzu deployed and enough resources for the TBS workload cluster A container image repository, such as Harbor, or Azure CR, etc, any compliant registry should do A place to run commands (a linux host is best IMHO) About the Tanzu Build Service There are several…
This is the whole post. (Is there an easier way to do this?) kubectl run --rm -i --tty busybox --image=busybox --restart=Never --overrides='{"spec": {"template": {"spec": {"containers": [{"securityContext": {"privileged": true} }]}}}}' -- whoami Boom! Now make sure you can’t do that in your cluster.
Problem : I need to package software Solution: Dockerfiles New Problem: I need to manage Dockerfiles New solution: ??? Overview There are many pieces to a modern, secure software supply chain. I say modern because this series of posts will focus on what it takes to build a secure software supply chain when the target for runnign these applications is Kubernetes. Kubernetes means containers…and…
First, let me say that Kubernetes is an extremely challenging piece of software to use, and, of course, to secure. I work at VMware in the Tanzu group and Kubernetes is a massive part of our portfolio–in fact it’s the base of almost everything we do. But it’s just the base. You have to add so much on top of Kubernetes to make it useful, and even more to secure it. But enough about that, let’s talk…
Today I figured I’d update my Linux workstation. Honestly I don’t like to do it because stuff breaks all the time. But I don’t want an out of date system either, so I just ran it ( eep ), and of course the update crashed midway through, and when I went to hard reboot I had a blinking cursor, which, from an existential perspective is awful and yet perfect at the same time. Thanks Linux! Haha.…