Recent content on Sander Knape
The first post in this series argued that AI is moving engineering up the stack, away from implementation and toward systems, architecture, and governance. The second argued that this changes what engineers do: judgment replaces implementation as the primary constraint, and the best engineers become orchestrators rather than builders. The third question—the one boards and executive teams most need…
The previous post argued that AI is moving engineering up the stack, shifting effort from writing code to designing systems and governing software in production. That has implications for strategy and organizational design. But it also changes something more personal: what it actually means to do this job. If implementation is no longer the constraint, what is? My argument is judgment. And if…
The debate about AI and software development is stuck on the wrong question. Most discussions ask whether AI writes good enough code, whether it matches human craftsmanship, whether it introduces bugs, whether developers can trust it. Reasonable questions. Increasingly beside the point. The shift that matters more isn’t about code quality. It’s about where engineering effort is…
Terraform users will likely be familiar with “data source will be read during apply” messages that may appear in the plan output. These messages can be confusing and may even lead to unexpected re-creation of resources. Typically, these messages are related to using data sources in combination with Terraform modules and explicit dependencies. Data sources and modules are two powerful…
Getting started with setting up proper monitoring dashboards for your application and infrastructure can be challenging. Where to begin? My typical answer to such a question would be to start with the “Golden Signals”. This blog post will dive into the golden signals and share how you can get started with these signals in Google Cloud using Managed Prometheus and the nginx-ingress…
There are many ways to improve the developer experience of deploying infrastructure into the Cloud. One such method is by shifting left: provide early feedback to shorten the feedback loop and speed up development. When deploying infrastructure into AWS with an infrastructure as code tool such as Terraform, you can validate that code as part of a CI/CD pipeline. A pull request can automatically…
GitHub Actions is a component of GitHub that allows you to create automated workflows. Through the many different events that can trigger workflows you are free to build whatever automation you want. While the most common use case is building CI/CD pipelines, the possibilities are pretty much endless. Check out this list of awesome actions to get some inspiration. Having spent quite a bit of time…
Serverless technology is getting more popular by the day. More and more people are starting to experiment with it and learn for which use cases it can add value. In this blog post I share an example of what a fully Serverless workflow can achieve. For a while now I’ve been curious how one would implement a payment workflow on a website. I was aware that platforms like Stripe , Adyen and…
The popularity of centralized platform teams is rising. The latest Puppet State of DevOps Report shows that 63% of the respondents have at least one internal platform. Platforms are vital enablers for a more DevOps way of working as they provide self-service capabilities that development teams can autonomously utilize. The definition of a “platform” isn’t set in stone though.…
Last year AWS released a new iteration of their API Gateway product: HTTP APIs . This new version promises lower prices, improved performance and some new features. Some features that are available in the older REST API are not (yet) available for HTTP APIs, though. The official comparison page gives a good overview of which features are available in both products. My favorite new feature…
There are a few reasons that I love my job. One of the most important ones is the variety of work. As a cloud/platform engineer, every day is different. Work goes from writing automation in some programming language, setting up a dashboard in a monitoring/logging tool, hardening Linux machines, writing Infrastructure as Code, building (standardized) CI/CD pipelines, giving workshops, analyzing…
Performing (automated) tests on pull requests is a powerful mechanism to reduce the feedback loop on code changes. Known as shift left , the idea is that the earlier you find an issue with your code, the easier it is to fix it. For one, as you wrote the code recently it’s easier to get back into it. And of course, any code issue that doesn’t hit production is another potential issue…
I’ve been blogging for close to four years now. I started blogging because I wanted to pick up a new skill (writing) and challenge myself to do something new. Because I blog about what is both my work and hobby - software development and more specifically, cloud/platform engineering - it’s relatively simple to come up with new subjects and I also grow my knowledge as I write these…
Last year November GitHub released GitHub Actions , a CI/CD solution build on top of GitHub’s Source Code Management. GitHub Actions is very convenient to use when your source code is already stored in GitHub as no additional tool is required for your CI/CD requirements. This blog is for example updated through a GitHub Actions workflow whenever I push an update to my GitHub repository (like…
Hosting static websites is great. As they only contain static assets to be downloaded by the visitor’s browser - think HTML, CSS, Javascript, Fonts, images - no server-side code such as Java or PHP needs to be run. They’re therefore typically faster to load than dynamic websites, they have a smaller attack surface, and are easier to cache for even better performance. That is why some…
The most exciting time of the year for AWS Enthusiasts is upon us. In exactly seven days, AWS re:Invent 2019 will kick off and everyone is excited to see what great features will be released and announced this time around. This year especially though, many new features are already released the weeks leading up to re:Invent. If you haven’t been paying attention, it was easy to much some great…
It hasn’t been that long since it was normal to request compute capacity at some operations department within your organization. In fact, it’s probably still pretty common in some organizations. With the move to virtualization and especially the cloud, this process of course has changed dramatically for the good. Not only compute capacity for applications, but also resources such as…
How do you properly use an SSH key in a Dockerfile? There are many ways to do it, including many ways to do it wrong. What you will want to prevent is that your ssh key ends up in one of your intermediate images or layers. These are the layers that Docker creates with pretty much every command in your Dockerfile. You may think that you properly clean up your secrets later in the Dockerfile, but…
The AWS Cloud Development Kit (AWS CDK) is a new framework for defining Infrastructure as Code (IaC) by AWS. It allows you to write IaC in a set of different languages. At the moment the following languages are supported: Javascript, Typescript, Python, Java, .NET. Support for other languages is coming. Of course, other methods like CloudFormation and Terraform already exist to write IaC. Using…
I recently blogged about automated deployments to Kubernetes using GitLab . One of the steps required when automating deployments is replacing the Docker tag with the correct value in the Kubernetes Deployment. In that blog post, this looks like the following: deployment.yaml apiVersion : apps/v1 kind : Deployment metadata : [ ...] spec : template : spec : containers : image :…
In recent years Kubernetes has quickly gained a lot of popularity and it currently has huge momentum. Adoption is rising while at the same time, new users find out the areas where Kubernetes is still lacking. One such area is the lifecycle management of application configuration. The construct in Kubernetes to store such configuration is the ConfigMap . These ConfigMaps can be referenced from Pods…
In this blog post we’ll go through the steps of creating an automated deployment pipeline for Kubernetes using GitLab. In the end we’ll have a simple Go application running that very excitingly returns “Hello, World!”. Prerequisites # Before we can begin our quest for automation, we’ll need to set up some tools. Many alternatives of course exist to the tools that I…
Yesterday, AWS announced the release of an important and much-wanted new feature for S3: blocking the creation of public S3 buckets on an account-wide . Enough has been written already about open S3 buckets on the internet. Given that it is very simple to create a public S3 bucket, we regularly learn about new (big) companies that have exposed privacy-sensitive data to the world through such…
It is still very common to develop an application locally on a laptop/desktop before pushing it to a production-like environment. The local development environment is kept as close as possible to production using technology such as Docker or AWS SAM when working with AWS Lambda. However, when working with AWS resources through Identity and Access Management (IAM) policies, local IAM permissions…
It’s been close to two years since I started working with AWS CloudFormation , the Infrastructure as Code solution by and for AWS. With CloudFormation you can describe and provision your entire AWS infrastructure in code. The entire AWS environment I work with is provisioned through CloudFormation - we only have read-only access to our AWS accounts. We now work with a number of different…
Serverless is a relatively new term. It’s a software development paradigm where the entire concept of a “server” is abstracted away from the development process. You essentially only use managed services that handle scaling, and you pay only for what you use. You no longer need to think about operating systems, security patches, scaling configuration and more. All this is handled…
AWS Secrets Manager is a service recently released designed to make the management of secrets easier. It provides built-in support for Amazon RDS, making it very easy to set and rotate secrets and use the CLI or an SDK to retrieve secrets from applications. Through the use of custom Lambda functions, essentially any database or an otherwise protected endpoint is supported. Setting up Secrets…
Secret management is one of those security topics that is often an after-thought while designing systems. Passwords are set up manually, shared through non-secure methods such as e-mail or Slack, and password rotation is often neglected because it’s time-consuming and error-prone. This is a shame because with some effort, secret management can definitely be automated. If done properly, less…
Serverless applications are everywhere these days. Having been introduced some years ago with the introduction of AWS Lambda, today serverless is much more then Function as a Service (FaaS). AWS is even starting to use the term in their products: at AWS re:Invent 2017 “Aurora Serverless” was introduced, a fully managed RDMS database. How do you build such applications? Given that you…
AWS API Gateway is a managed service from Amazon Web Services that allows you to easily create an API endpoint. An incoming request can either be forwarded to an HTTP endpoint, can invoke a Lambda function, or connect to an AWS service. Arguably, the Lambda integration is most commonly used as it allows you to invoke a piece of code without managing a server or operating system (thus, combined…
CloudFormation is the AWS product for Infrastructure as Code. It allows you to provision AWS resources through a template that describes how to configure that resource. Unfortunately, CloudFormation will sometimes be behind on new features released by AWS. Where the AWS console and API will allow you to deploy resources with a certain configuration, in CloudFormation specific settings might simply…
I’ve written about serverless architectures before ( a serverless website in S3 and dynamic image generation with Lambda ). I strongly believe in this notion as it minimizes required maintenance and makes the initial setup much easier. Of course, there is no silver bullet for every problem but use cases definitely exist where a serverless architecture makes a lot of sense. One such use case…
Yesterday, AWS released CloudFormation StackSets . A StackSet is a set of CloudFormation stacks that can easily be deployed to multiple AWS accounts and/or multiple AWS regions. Before, each stack had to be deployed separately and custom scripts were required to orchestrate deploying to multiple accounts/regions. Therefore, this feature is bound to make the lives of AWS administrators a bit…
ReactJS is a Javascript library for building user interfaces. The way it ties in with HTML makes it very easy to create interactive components that can easily be reused among different locations within your application. Typically, people build Single Page Applications (SPA) using React. This way, much of the logic required to display a webpage is brought to the visitors’ browser. Instead of…
Maintaining a web server for your website is not ideal. It’s tedious, takes a lot of time if done properly and distracts from what you really want to do: sell a product, share information, e.t.c. The more time and money you spend on the platform, the less is available to spend on the website and your core business. Unless you’re a hosting company of course, but then this blog post…
Infrastructure as Code (IaC) is a very powerful concept. The idea is that you put all infrastructure resources - networks, subnets, load balancers, firewalls and so on - in code. You then deploy your infrastructure the same way application developers deploy their code: through a continuous integration / continuous deployment (CI/CD) pipeline. Other benefits already reaped by application developers…
The combination of API Gateway and Lambda is very powerful. It allows you to build some complex functionalities without maintaining any virtual machines yourself. Lambda can be hooked up to many other (AWS) Services including DynamoDB, Kinesis and S3. This paradigm, better known as serverless , is truly upcoming for years now and its certainly time to jump on the bandwagon if you haven’t…
Did you ever write your own authentication service? In essence it’s quite trivial: allow a user to enter a username and a password. Next, look in a database for a row/document that matches the received data. If found, login. If not, be gone. It becomes harder when you start thinking more about proper security. And what about features such as password resets, login throttling or logins with…
Are your SSH log files flooding with failing login attempts? I’ve seen many questions on websites such as Stackoverflow and Stackexchange from worried people that someone is actively targeting their servers with brute-force password logins attempts. Let me get one thing straight first: you are not special! It’s part of internet life: many botnets constantly attempt to login to servers.…
I have been running my blog for a few months now on a DigitalOcean droplet. Apart from the few metrics DigitalOcean provides, I never really had any insight into how my server is performing. This includes metrics like CPU and Memory, but also metrics from the applications I run to host my blog, in particular Nginx, php-fpm and MySQL. To get more insight into these metrics, I decided to install the…
Do you write integration tests? What about unit tests? I believe that more people say “Yes” to the second question than to the first. Which is kinda weird - for many applications, it really isn’t that hard to write integration tests. It might not even be necessary to setup your own infrastructure to run these tests. Many CI tools these days allow you to install databases, queues…
If you maintain your own server or servers with SSL certicates, you might know how annoying it can be to keep those certificates up to date. Especially when you have a multitude of servers, some possibly even serving the same certificates, this is a hassle to manage. Luckily, there is a solution, and its called Let’s Encrypt. Introducing Let’s Encrypt # Let’s Encrypt is a…
Nomad is a distributed, multi-datacenter scheduler for containers, virtual machines and more. It’s a tool from Hashicorp, the company that also brings us Consul, a service-discovery tool that allows you to register and discover services. With the latest big release of Nomad (version 0.4), integration with Consul is improved which promises to significantly simplify the creation of a…
Setting up SSL for your server may seem like a daunting task. In addition, why would you do it? What are the benefits? There are multiple, actually, with some of the most important ones being: It’s better for SEO. Back in 2014, Google announced they would start with giving HTTPS websites a little boost in the search results. It’s not slower than HTTP. In fact - it will even be faster with HTTP2…
I’m Sander Knape, a Staff Platform Engineer at Stack Overflow. I work primarily on developer experience: helping engineers get from idea to impact faster, with less friction along the way. A lot of my current work sits at the intersection of platform engineering, AI, and engineering effectiveness. Before that, I worked as a software engineer, cloud engineer, SRE, and tech lead. That…