Persistent Volumes provide a means of connecting a service with storage. Local SSDs will, of course be uncontested in terms of latency, but iSCSI allows a container to be scheduled on any compute node. Local Disk iSCSI NFSv3 Volume Claim Required yes yes no Authenticated – yes no Schedule on any Node no yes yes Concurrent Access yes no yes Native Linux File System yes yes no Uses Buffer…
These are the steps to diagnosing a problem booting OpenBSD using EFI and PXE. probing: pc0 com0 mem[636K 1887M 16M 84K 88K 752K 16K 4M 584K 7M 6M 6148M] disk: hd0* hd1* hd2* hd3* hd4 >> OpenBSD/amd64 BOOTX64 3.69 boot> cannot open tftp:/etc/random.seed: No such file or directory booting tftp:/bsd: open tftp:/bsd: No such file or directory failed(2). will try /bsd
QEMU is perhaps best known for the ability to emulate various architectures, but it also has wide support for the native Linux and Mac hypervisors. In this way it is similar to using Bhyve .
For large data sets, home directories, or backups, ZFS has a wide range of supurb features. But if an advanced file system is not available, as FFS on OpenBSD or EXT2/3/4 on Linux can also be used. For backups the goal will be to maintain three copies that guard against hardware failure and human error. Home directory Primary backup Secondary backup
Among the brilliant features of WireGuard is the ability to classify traffic based on peer public keys . This means that packets can be filtered by interface or require that each peer use a specific tunnel address.
If a local network is secure, we can easily configure a local registry without having to manage TLS certificates $ kubectl get svc -o wide registry NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR registry ClusterIP None <none> 5000/TCP 5d20h app=registry
After bootstrapping up a basic Kubernetes Lab , a dynamic routing protocol provides effective routing at all layers because each router and switch understands how to route traffic effectively # ospfctl show rib Destination Nexthop Path Type Type Cost Uptime 192.168.0.4 192.168.2.4 Intra-Area Router 20 04:47:00 192.168.2.0/24 192.168.2.3 C Intra-Area Network 20 05:06:25 10.244.1.0/24 192.168.2.65…
Also known as pivot tables , distributing categories across colums is a very intuative way to display query results. Some built-in methods exist in PostgreSQL The psql \crosstab command provides a quick way to reformat the output of an interactive query, but is of no use for applications. The crosstab functions from the contrib module are awkward to use since they take a query as input, and has…
I first configured IPv6 Internet access at my home in 2003. More than 20 years later IPv6 is widely available, but remains a challenge to implement on a local area network. Perhaps the most interesting question is not how to implement IPv6 networking, but determining why anyone would need it. The 128-bit address space is divided into a strict hierarchy, and it may be that this allows the protocol…
awk is a processing language with a man page that can be printed on eight sheets of paper. Most importantly, it is a standard part of any Unix-like environment, so it can be used in contexts where the features of a target installation are unknown. Program structure has three sections, all of which are optional BEGIN { ... } /patternN/ { ... } END { ... }
In my view, the most significant feature missing from PostgreSQL is the ability to create views without hard dependencies. Views reference objects by oid not by name , hence creating a view always has the potential to break automated schema updates.
ZFS has a truly unique combination of features ZFS is able to distribute data across multiple devices, and this format is portable. Unlike traditional RAID, only dataset records are copied, making recovery very efficient. ZFS does away with the need for a volume manager, so there are no platform-specific layers such as BSD disklabels, or LVM. ( BTRFS also has these capabilities, but is…
For many reasons it is useful to store arbitrary nested key-value and array data in JSON. Among the deficiencies of JSON is limited data types, but the great benefit is that it is universal can be queried.
Puppeteer is a Node.js library which provides an API to control Chromium using the DevTools Protocol. JavaScript may be executed in the context of the headless browser, so any kind of change may be made to the page before taking a snapshot (PNG) or saving as a PDF.
Declarative partitioning in enables the PostgreSQL query planner to optimize queries in several ways, most notably by pruning partitions that are irrelevant for the query. This results in tremendous performance gains, but only works for values known during the planning stage . If the partition key is also a natural key, such as a date, pruning may appear to work in all cases, but if an…
For FreeBSD, the ability to PXE boot BIOS and UEFI systems is not new, but documentation frequently assumes an NFS mounted root file system. In my view NFS is unworkable since file access is not logged, and a hung mount is difficult to diagnose. By using TFTP to pivot to a minroot, then switching to HTTP the entire process can be audited and tailored for individual hosts. This methodology aims to…
One of the most useful skills a Systems Engineer employees is a method of keeping personal notes. This serves as a form of documentation that is constantly revised that captures current knowledge of a system and the tricks to accomplishing specialized tasks effectively. Note taking also services to keep long-term goals moving. On a daily basis write-ahead logging can provide short-term focus. By…
In software or systems engineering, the ability to spin up virtual machines is a valuable capability. Virtual machines provide a means of testing software or configuration in on multiple platforms, and in multiple configurations. In production, a high-performance and reliable hypervisor allows services to be provisioned and updated one at a time. This capability avoids the added risk and…
These are operations that I commonly use to aid supporting PostgreSQL databases.
Ubuntu supports major version upgrades using the do-release-upgrade utility, but in my experience this is not a reliable means of updating the operating system. Automating installation facilitates testing and iterative design of systems configuration. Although Ubuntu is a derivative of Debian, it doesn't follow the same conventions for an automated install. x86_64 hardware and hypervisors usually…
Any time a task needs to be repeated it is a good candidate for automation. Any time a task needs to be reproducible automation is required. For running frequent experiments on my home lab, and the ability to quickly rebuild a machine is of immense benefit.
sidecomment.io is a service that allows readers to select text and create a ticket with suggestions or corrections that the site owner can respond to. Like a commenting platform such as Hyvor Talk , JavaScript is included from an external CDN, and is authorized by a site code . This concept developed after I noticed how many people were willing to take the time to write an email to point out…
I first posted my notes on using OpenBSD as my workstation in of 2013, but I probably switched from Arch Linux in 2005. This is a collection of customizations I have used for over the years, as well as some hints for using some more modern features.
JSON Web Tokens (JWT) provide a standardized representation of claims to be transferred between two parties . This payload may contain any JSON data, but for web services this is most commonly applied to stashing some bit of data that identifies a user that has passed a prior authorization step. Keeping the length of the encoded string short is important to minimizing overhead, especially for…
Modern web browsers all allow REST interfaces to external services depends on using a request-response standard called Cross-Origin Resource Sharing. This mode is available by setting the mode attribute of a JavaScript Request object. The following is an example using the Fetch API async function sct_postData(url, data= {} ) { const response = await fetch(url, { method: 'POST' , headers: {…
Some virtual private server providers support OpenBSD as an install target. This is a wonderful start, but only a start because the image applied and resided will probably not a useful partition scheme and mount options that makes sense for a given security and operational strategy. Fortunately, a hosting provider such as Vultr can deliver an OpenBSD instance that can reinstall itself using and…
There are two reasons I have found it advantageous to synchronize local user accounts to an SQL lookup table: Databases frequently keep records based on the user who ran a task; a users table allows you to write queries that include or exclude certain groups Applications need to filter records using access controls that are consistent with file system permissions Arrays provide a very natural…
SQL databases provide a critical service by giving the application globally consistent access to data, but logical replication makes it possible to trade in the guarantee that a query always reads the most recent write for other features. Instead of global behavior, logical replication allows a relational database to target particular features for distributed processing. I may be mistaken, but I…
The PostgreSQL server does not have a feature called "autocomit", but the most popular database library for Python, psycopg2 has it enabled by default . I say "enabled", but to disable this behavior we have to turn something on conn.autocommit = True The psycopg2 documentation has a warning about the default behavior Warning: By default, any query execution, including a simple SELECT will start a…
Haml is an templating language for writing HTML. Over the past twenty years the capabilities of HTML+CSS have become good enough for nearly any kind of document to be rendred in a Web browser. I use Haml for all of my writing. Haml combines a set of valuable features: A notation that mirrors HTML Inline scripting and control flow Blocks of text can be interpreted using custom filters Haml 6+…
OpenBSD has a very good story for automated installations, which includes features such as: Boot parameters set on boot (example: switch to serial port) Configuration of network interfaces Partitioning & adaptive disklabels Selection and installation of base sets Custom site package that can contain arbitrary content Installation of non-root user with SSH public key Applying patches on first boot