Rambling on software.
import Activity from "@components/Activity.astro" import Gallery from "@components/Gallery.astro"; import p1 from "./p1.jpg"; import p2 from "./p2.jpg"; import route from "./route.json" About a month ago we ran our first Ultramarathon, the River Run 50k. We ran the Missoula Marathon last year; I remember thinking how impossible a marathon sounded at that time. 26.2 miles. But, my goal since…
Managing a database is more than just storing information—it's about storing data efficiently, effectively, and balancing trade-offs. We can employ techniques like data normalization to optimize our data for maintainability and readability, or data denormalization to optimize for raw query speed. Data normalization is the process of breaking data down into structured tables to reduce duplication…
I recently ran into an issue with our Mac build machine in AWS, where the EC2 instance kept running out of disk space. It should be easy enough to bump the EBS volume size and call it a day. Wrong. No matter what I changed the volume size to, the Mac instance would not recognize the new space. What gives? After some digging, I found that Mac requires us to resize the OS APFS (Apple File System)…
As developers, querying in PostgreSQL for hierarchical data is difficult. SQL is a declarative programming language, but our brains are trained to think imperatively. Recursive queries using Common Table Expressions (CTE) simplify writing iterative queries and are essential in traversing hierarchical data, like tree or graph structures. Throughout this article, we'll explore SQL recursive queries…
You may have heard that "data is the new oil." By itself, data is unrefined and not valuable, but given processing and refinement, it becomes precious. We gain insights into our products, applications, and customers by exploring our data. PostgreSQL exposes aggregate functions that give us the tools to transform and process our data to provide meaning. In this article, we'll take a look at how to…
Foreign keys in PostgreSQL are a fundamental way of defining relationships between tables. Splitting data across multiple tables is foundational to relational databases and data normalization. However, when spreading related data across various tables, we must ensure that values in one table are valid and exist in another. Enter foreign keys.
EXPLAIN is a command we prepend to queries to tell us what operations PostgreSQL plans to run for that query, how many rows each operation touches in the database(s), and how long those operations take. EXPLAIN is the go-to performance diagnostic tool for discovering what is causing a query to run slowly.
Databases store an application's most valuable asset: data. This data includes personal information like names, passwords, and street addresses, but it could also include company secrets. Having an insecure database is devastating for a few reasons: Leaking personal data erodes trust from users and customers Falling share prices or loss of revenue You can be hit with regulatory penalties (e.g.,…
Common Table Expressions (CTEs) are my go-to tool when writing complex SQL queries. They can be thought of as working like sub-queries or temporary tables but with their own superpowers. In this article, we'll take a deep dive into what CTEs are and how they can simplify your queries, how recursive CTES can turn impossible PostgreSQL queries into possible, and how to use CTEs to create…
Tablespaces are a PostgreSQL feature that gets easily overlooked but can provide substantial speed improvements and cost-saving potential. As our database grows, tablespaces become a powerful tool in our toolbelt.
How we design and lay out our tables is one of the most important decisions in developing effective and efficient PostgreSQL databases. A database schema is the architecture or structure of the data in our database. It defines what tables we'll have, columns, data types, naming, and relationships between tables.
Imagine you have a massive amount of time-series data you want to explore and visualize. Seeing the latest trends, the historical patterns, and the outliers in your data can help you gain insights and make decisions. But how do you visualize and analyze time-series data effectively? How do you create graphs, plots, and other visualizations for real-time analytics showing the current state of your…
import { Accordion } from "../../../../components/Accordion"; When writing articles about PostgreSQL there are times I need to generate large swaths of data to test and demonstrate a topic (for example, these Timescale articles here and here ). It can be a pain to manually insert data or create a CSV and import it into PostgreSQL--which I always have to StackOverflow. Lately, I ran into a much…
I worked with Timescale again to write an article on how table bloat effects large tables, and how we can fix it. We cover a range of topics including: What is table bloat? Why is it an issue? How to monitor table bloat? What Vacuum and AutoVacuum is Tips with tuning AutoVacuum Table Partitioning How Timescale chunks makes partitioning easier And more!
I had the pleasure of working with Timescale to write an article on how to reduce your PostgreSQL database size. We cover a range of topics including: What are pages and tuples How table bloat occurs Why we should vacuum our tables Timescale compression Retention policies And more!
import { Image } from "astro:assets"; import baseGraph from "./base_graph.svg"; import table from "./table.svg"; import unidirect from "./unidirect.svg"; In Postgres: The Graph Database You Didn't Know You Had we covered how we can store a DAG (Directed Acyclic Graph) in a relational database. We looked at one representation of a graph data structure called an adjacency list--a table with two…
import { Prompt } from "./prompt"; Tools like ChatGPT and Bing have been skyrocketing in popularity. ChatGPT took five days since launch to reach 1 million users . Governments all over the world are trying to figure out what to do with this technology . Sweet AI-generated images are showing up everywhere; like this dog I generated using Bing: "Can you generate a picture of an astronaut dog jumping…
Debugging is an essential skill for any software developer. It allows us to find and fix errors in our code, improve quality and performance, and learn from our mistakes. One of the most common and powerful tools for debugging Python code is the Python Debugger (PDB) . PDB is a module that provides an interactive command-line interface for inspecting and manipulating the state of a running…
PostgreSQL (Postgres), is a powerful relational database that can store a wide range of data types and data structures. When it comes to storing graph data structures we might reach for a database marketed for that use case like Neo4J or Dgraph. Hold your horses! While Postgres is not generally thought of when working with graph data structures, it is perfectly capable to store and query graph…
Have you ever found yourself in the middle of coding a feature, only to receive a notification that production is down? If you're using Git, you now have a few options: commit your work, stash your work, or discard your work. Each of these options has its own set of problems, depending on the situation. If you commit half-baked code, you may need to come back later and rebase to clean up your…
A few months ago I had an opportunity to give a presentation to soon-to-be Computer Science (CS) graduates. We presented the software development process ( SDLC ), and how different teams at Schweitzer Engineering Laboratories adapt the development process to their needs. We ended the presentation with tips for what to expect entering the industry. There is a big disconnect between what we learn…
Code dependencies are like Lego's . We are able to pull in other people's code, combining and stacking different packages together to achieve our goals. Using dependencies greatly reduces the complexity of developing software. We can take advantage of the hard work someone has already done to solve a problem so that we can continue building the projects we want. A development pipeline can have…
CSS is one of those technologies that has a low barrier to entry (good thing 🎉), but because of this sometimes how it works can seem like magic. It's easy to get started writing CSS, so we dive head first into it. Sometimes we smash our face into the concrete with frustration. Why won't my text move over yonder? Where did that scroll bar come from? How do I center this junk? This is a series I'm…
Our default response to any idea that comes up should be: “Interesting. Maybe some day.” In other words, a very soft “no” that leaves all our options open. We don’t put it in a backlog. We give it space so we can learn whether it’s really important and what it might entail. "Shape Up" - Ryan Singer (Basecamp) Here's a story: At a previous employer, I had a boss who I thought was pretty tough. Not…
import { Image } from "astro:assets"; import htmlDiagramImage from "./html-diagram.svg?url"; import reactRenderingImage from "./react-rendering.svg?url"; React does a fantastic job abstracting away the rendering of webpages. When state changes, our UI's reflect it. But, have you ever gotten done writing your application just to notice things seem off? Clicking a button displaying a popover…
This is an attempt at a different kind of post. Something less technical. Something more personal. I'm a pretty big music fan, and I'm always scrounging around the internet for new artists and songs to fill my Spotify portfolio. One of my biggest fears is being that 50 year old dude who still only listens to those same two Def Leppard songs. Mullet optional. Just to be clear, this isn't a list of…
You need your nausea. It is a message. It will tell us what is wrong with you. Lost Connections is a challenge to the classic story any of us with depression and anxiety have heard. Go to a doctor. Get anti-depression pills. Become normal . Johann doesn't completely remove that fact that medical anti-depressants work, they do; just for a smaller subset of humans than is currently being prescribed.…
Many sites try to avoid our ad blockers through limiting content or just straight out pleaing. Other companies are looking at encrypting any kind of indicator that screams "AD!" through obfuscation and other fancy techniques--confusing ad blockers. Here we'll look at some interesting techniques Facebook deploys to ensure we view their ads. What does a Facebook ad look like? Facebook ads take the…
We can all agree that writing clean code is important. It makes it easier to work in a team, and even if we're a single developer, clean code keeps our future selves happy. Ever look at your old code and wonder, "What is this crap?" If you said "no", write more code--the feeling will come. For those who said "yes", let's look at some techniques that will improve your code readability. Use Good…
One motivating factor in using a terminal is the productivity gains... that is, once you can sweep the keyboard like an 80's metal guitarist. Getting a few basic keyboard shortcuts/commands down can drastically improve the usability of the terminal. Let's take a look at a few of my favorites. (I'll be showing off Git Bash in this article, but these commands work in most terminals) Movement Move to…
This post comes from an example I presented at Boise Code Camp 2019 comparing imperative and functional ways of solving problems. The goal was not to teach functional programming in its entirety, but introduce a different way of thinking that is outside of traditional approaches (loops, mutation, etc). Having a different frame of reference adds more tools to the toolbox when faced with a problem.…
The latest version of a React component I maintain just released, v2 of react-scroll-to. First, it started out as a component I needed, but over time a few people started using it. Because it was initially created for my specific use-case it started to show its warts when new features were ask for. The goals for v2 were two-part: introduce a more flexible API and implement some highly asked for…
key , the property we throw some value into to suppress warnings. This property is used by React to determine when a component is deleted, updated, or created. We generally don't interact much with the key prop--except in loops. My team recently ran into an issue lately--how do we rerender all child components when a state has changed? The simplest approach... you guessed it, key ! Let's take a…
FabricJS is an awesome library that handles Canvas interactions and rendering. Inherently it is a pretty performant library, but recently we ran into an issue where we needed to render ~8mb of json data (10k objects). Given any framework, it is no easy task to render this much data. The biggest culprit while rendering a lot of objects is renders (or rerenders). When the canvas has to show any…
When working with React we get introduced to state as a way if storing component's data. There is another way that generally gets overlooked: class properties. Example class AnnoyingButton extends React.Component { intervalId = null; // Here! id = generateUniqueId(); // Here! state = { isDisabled: false, }; componentDidMount() { this.intervalId = setInterval(() => { this.setState(({ isDisabled })…
In software development, we can easily get trapped in analysis paralysis . Having the foresight to make all the correct decisions is hard. We make hundreds, if not thousands, of decisions during the lifetime of our software, and we're likely to get something wrong. Instead of trying to get every minutia correct before starting a project, we should aim to write software that can evolve. Continuous…
Last time we went through using Typescript with React, and setup a small counter application. A lot of applications out there use Redux. A common pain-point for me was how to setup Typescript with Redux--it's a common hurdle with my team also. Though Redux is a totally different library, it holds a big share of the state management market in React applications. In part two we'll take a detour from…
Typescript recently released generic type arguments for JSX in their 2.9 release. It's a mouthfull, but what does that mean for us? A common use-case I'm excited for is allowing consumers of libraries to extend a component's props. Using [dynamic components]({% post_url 2017-07-26-injecting-react-tag-types %}) we'll look at allowing our components to be extended even more. What Are Generic Type…
Finished Example: Click Here Learning Typescript with React can be challenging, and in my case, cursing Microsoft and throwing any on every variable. The goal of this series is to document what I've learned while developing applications in React, and trying hard to shove in Typescript. Like many things, the initial learning curve may be staggering, but adding a type-checker repays dividends once…
Beware the mega component. Mega components contain around 50 props--when the stars align, only certain values produce specialized output. For example, let's assume we have a <IdentityInput /> component which provides user login using only a fingerprint. Nifty, eh? Our component has a few properties: <IdentityInput onSuccess={login} onFail={showError} fingerprint={value} humanOnly /> allowing for…
Introduction Often, I find myself looking through open-source projects to see how they solve issues. Not bugs, but problems in code that cause repetition or excessive boilerplate. Modals have given me problems, specifically when having to create a specialized modal for different workflows. They are awkward. Modals, in the DOM, live outside of the scope of the current workflow. And, part of their…
import Alert from "../../../../components/Alert.astro"; Update (3/20/2023): Since writing this post the community has called this technique "Render Props". There is a technique for creating reusable components that got missed in my last post, Tips on Creating Reusable Components , that I'd like to talk about. That is, components that have functions as children. This is a great tool that mirrors…
Introduction A huge selling point of React is its use of composable, reusable components. Everything is built off the idea of V = F(d) , or view/UI is created by some function acting on data/state. How do we create more advanced user interface? Well, just add more functions to the party (For example V = G(F(E(S(d)))) ). That's cute, why does this matter? Thinking of our components as functions,…
Introduction Think of Higher Order Components (HOC) like Pacman. As Pacman consumes his smaller looking dot counterparts, Pacman gains superpower abilities. Like ghost eating! HOC's work similarly... minus ghosts. A HOC consumes a base component, applies some new ability or super power, then spits out a new modified base component. To help show the power of HOC's, lets create one that does a…
Introduction Let's say we're working on an UI component library using React JS. We make a super awesome looking button, maybe even the best button in the world. But suddenly, our computer shuts off without saving our component! Like Tenacious D, we create a tribute to the button which looks like: class AwesomeButton extends Component { render() { const { children, ...other } = this.props; return…
This post is part of the archive. These are mainly random posts that were written throughout the years in no coherent order. FlightJS FlightJs isn’t the new kid on the block. This little framework has gone unnoticed in Github-land for years. Taken from the FlightJS Github: “Flight is a lightweight, component-based JavaScript framework that maps behavior to DOM nodes.” Reworded, we use reusable…