RSSAmplifier

Blog

Nik Ogura

Platform engineering, security, infrastructure, and the occasional philosophical tangent. Writing about Kubernetes, observability, GitOps, and building things that work.

nikogura.comRSS feed ↗45 posts

Latest posts

Your Hiring Pipeline Has the Same Bug as Your Deploy Pipeline

A résumé parser matches tokens, not competence. It's the same mistake as infrastructure that describes state instead of maintaining it, and the same mistake as confusing 'can' with 'does.' Keyword-gated hiring optimizes a proxy and quietly filters out exactly the senior people it claims to want.

Load-Bearing Humans

A system that only keeps running because a person is continuously holding it up is not working. The person is working. The system is on life support. 'Working' is one more overloaded word, and most of the time it quietly means 'works as long as the right human is standing there.'

Nice People Who Give Us Money

We are not paid to solve interesting engineering problems. We are paid to make products that make customers happy, and a customer is a nice person who gives us money. Under that definition, monitoring, testing, deployment, security, and resilience are not 'non-functional requirements.' They are the function.

Gambling on Failure

Most people gamble on success — they assume the thing will work, and they're genuinely surprised when it doesn't. A tiny, birdlike kung fu master taught me to gamble on failure instead. Expect every move to be blocked, and win anyway. It turns out to be the same discipline that keeps systems alive at 2 AM.

DDCRI: Declarative, Deterministic, Continuously Reconciling Infrastructure

What's in git is what's in your infrastructure — or alarms are sounding. DDCRI is the discipline that makes that sentence literally true — FluxCD, Kustomize, Crossplane, and Upjet reconciling a control repository continuously, with drift wired up as a pageable condition. A canonical, example-driven walkthrough.

Stop Holding Out for a Hero

Incident response is either an engineering discipline — measured, quantified, repeatable, owned, evaluated — or it is a craft a few heroes practice and nobody else can duplicate. Heroes are great. You shouldn't need them, and you shouldn't bet the company on still having them.

Don't Paint Yourself Into a Corner

Larry Wall built Perl around a principle: no unnecessary limitations. Most of the limitations we build into our own code aren't necessary either — they're laziness wearing the costume of caution, and every one is a wet patch of floor between you and the door. Stop boxing in your future self.

Most Infrastructure as Code Is Broken — and Reconciliation Is Only Half the Reason

Run terraform plan against infrastructure nobody has touched in a month and watch it propose changes. That drift is the absence of a reconciliation loop. But the missing loop is only half of why most Infrastructure as Code is broken — and bolting a loop onto the other half just gets you to broken faster.

There's More Than One Way to Get Observability Right

The specialize-versus-unify argument feels like a religious war. It isn't. Both sides are right — they're answering different questions. There are several ways to get observability right. The way to get it wrong is to never ask which one you're building for.

Continuous Acceptance Tests

An acceptance test run once before deploy proves the data was correct for one instant. The data does not stay correct because the deploy was green. Stop retiring your best test the moment it passes. Run it forever.

Put Dex In Front of Google OAuth

Google OAuth has two surprises that make every internal-service auth story uglier than it should be. The standard workaround involves domain-wide delegation and a service account JSON key shipped to every application that wants group-based authorization. There is a much better answer that doesn't require any of that.

Incident Management

The role-based incident-response model Jesse Robbins brought from the fire service into web operations, written down from memory as I learned it. Incident Commander, Scribe, SMEs, severity, the CAN format, and the discipline that makes the framework actually work when the page goes off at 3 AM.

C-Style Thinking vs Go-Style Thinking

You can identify a programmer's native language by the tools they build. Go and Perl natives build tools with sane defaults that work out of the box. C and C++ natives build tools that require you to understand the entire problem space before you can do anything at all.

'Can' vs 'Does'

The difference between a system that can fail and a system that does fail is time. Murphy's Law is not a joke. It is a design constraint. Every moving part you add is another bet against the house, and the house always wins.

Control Repositories

A control repository is a git repository whose contents declare what should be running, intended to be reconciled by an automated controller. It looks like a source repo, but the discipline is different — reviews focus on what the system should be, duplication is often intentional, and the head of main is the state of the platform right now.

Trunk-Based Development

Long-running branches are an anti-pattern. The longer a branch is open, the more expensive the integration becomes, the less informative the diff, and the further the branch drifts from whatever is actually shipping. Trunk-based development is the alternative the continuous delivery community converged on two decades ago, and the evidence for it is overwhelming.

Web3 Is Just Infrastructure With a Hoodie

Everything in Web3 is something you already know. Wallet signing is SSH authentication v2, smart contracts are RPC endpoints with public immutable code, and the trust model is the same "verify, don't trust" principle that drives every good infrastructure system.

"Design Me a Highly Resilient Database"

There is no such thing as a "highly resilient database" in the abstract. The right answer starts with understanding the problem — the data, the product, the failure modes, the regulatory environment — not a product name.

Security Is Infrastructure

Security and infrastructure are not two disciplines that happen to overlap. They are one discipline that companies have artificially separated because org charts demand clean boxes and job postings demand clean titles.

Metrics, Logs, Traces, and Events: What's Actually Different

Four observability signals that get thrown around interchangeably. Understanding what makes each one distinct and where they overlap determines whether your observability stack scales or collapses under its own weight.

Distributed Tracing: A Practical Guide

Distributed tracing captures the complete journey of a single request as it passes through multiple services, enabling latency analysis, error propagation tracking, and root cause analysis across complex architectures.

Prometheus and OpenTelemetry: How They Fit Together

OpenTelemetry does not replace Prometheus. They solve different problems, they are converging, and understanding the boundary between them will save you from expensive architectural mistakes.

Puppets and Octopi: Why Top-Down Orchestration Hits a Wall

Centralized, imperative orchestration requires centralized coordination, and centralized coordination is a bottleneck that doesn't scale. Distributed, declarative convergence pushes intelligence to the edges.

The Best Dog Trainer in the World - Or Why Getting Better Isn't Helping

When something has been failing for a while despite competent people working on it, the problem is almost certainly not competence. Before you optimize, ask yourself whether you are training a dog or a cat.

FluxCD vs ArgoCD: Architectural Comparison

A deep comparison of the architectural differences between FluxCD and ArgoCD for experienced Kubernetes platform engineers, covering CRD design, state management, controller models, and where each tool wins.

GitOps

GitOps is not just keeping YAML in git. It is a specific operational model where a controller in the cluster continuously reconciles actual state against desired state declared in a git repository, providing audit trail, reproducibility, drift correction, and safe rollback.

GitHub Actions Reference Implementation

A reference implementation for GitHub Actions CI/CD pipelines covering automated testing, linting with golangci-lint and namedreturns, semantic versioning, automatic releases, and caching strategies.

Shell Functions

A collection of useful shell functions and aliases for Kubernetes debugging and daily operations, from context switching to JWT decoding to Prometheus config extraction.

Engineering Standards

Security, reliability, and compliance are non-negotiable. A comprehensive guide to Golang coding standards, testing practices, infrastructure patterns, observability, and the philosophy of craftsmanship in software engineering.

Cross-Cloud Kubernetes Clusters with AWS IRSA and Talos Linux

How to provision Kubernetes clusters on Talos Linux with OIDC-based AWS IRSA integration using Terraform, enabling cross-cloud workloads that authenticate to AWS services without static credentials.

FITFO - Figure It The (Fun?) Out

A reflection on how learning to program as a small business owner built the FITFO skill -- the ability to figure things out from scratch -- which became the foundation of a DevOps career.

Golang Design Tips

Many of Golang's unwritten rules can be summed up by the Go Proverbs. Here are practical extensions and recommendations for writing idiomatic, maintainable Go code.

Auto Updating AMI's on a Rolling Window with Terraform

A Terraform pattern for automatically selecting the newest AMI that is at least two weeks old, enabling a rolling window approach to AMI updates that avoids deploying untested images to production.

The Documentation Problem

Documentation is like pizza -- when it is good, it is really good, and when it is bad, it is still pizza. A perfect documentation system makes it more likely that docs will be written and maintained than not.

Vault Operator Notes

Operational notes on running the Vault Operator in Kubernetes, covering CRD configuration, secrets management, authentication testing, and Cert Manager integration.

Coding Standards (especially in Golang)

This is about design philosophy and how to approach your work. Every line of code should be to a standard you would be proud to publish under your own name, because git is forever.

TDD (Test-Driven Development)

TDD is not just about writing tests before code -- it is about alternating between tests and code so that you write code in discrete, testable units and capture every insight as an automated check that lives in your codebase forever.

Managed Secrets

Secrets management is an important job, but it sucks. A look at why UX is the key to a successful secrets system, and how Managed Secrets provides a YAML interface to your secret storage backend.

Using CircleCI as if it was a Maven Repo

A trick for using CircleCI 1.x build artifacts as a Maven repository by fetching dependency artifacts and installing them into the local Maven cache.

Dynamic Binary Toolkit: Tools that automatically keep themselves up to date!

A system for building self-updating command-line tools that automatically download, verify, and replace themselves with the latest version using checksums and GPG signatures.

Access and Identity that Just Works

A hands-on guide to spinning up an n-way multi-master LDAP directory for centralized access and identity management, covering server setup, replication, ACLs, and schema configuration.

LocalEnv

A simple trick to link your IDE terminal with your project's virtual environment, so command-line tools automatically use the same Python and libraries as your IDE.

One Shot OpenStack Liberty Installer

A single script that creates a full OpenStack Liberty cloud from nothing, demonstrating an infrastructure-as-code approach to cloud deployment.

Python Development on MacOS

Potentially timesaving tips and tricks for Python development on MacOS, covering virtual environments, Homebrew setup, and IDE configuration with JetBrains tools.

IAM Beyond AWS or Hacking Hacks, and the Hackers who Hack Them

How I built a system to impersonate the AWS metadata service on developer laptops, providing passwordless, expiring STS credentials linked to user identities via SSH key authentication.