Recent content on knrdlog
Install Podman sudo apt install podman podman-compose Compose can be called as podman-compose with autocomplete. Allow binding to privileged ports (<1024) echo 'net.ipv4.ip_unprivileged_port_start=1' > /etc/sysctl.d/100-podman.conf /usr/sbin/sysctl --system # apply setting Start containers after reboot without user login loginctl enable-linger $USER Pull images from Docker Hub without explicit…
Every internet connected server is subject to automated scans and probes from various parties. On web servers they typically try to find exploitable installations by calling e.g. ///wordpress/wp-admin/setup-config.php?step=1 . Another annoyance are web scrapers farming content for AI bot training and often enough ignoring therefore the robots.txt. These requests form a constant background noise…
The straight forward way to access a remote network is to setup a VPN connection with the router. If that’s not possible, a separate device is required to act as VPN client. A smartphone is particularly suitable for this as it is always on, small, has a low power consumption and makes it easy to configure a wifi connection with the network. The PinePhone is especially suitable for the job as…
Async Python Async frameworks are nowadays the standard for web development with python. FastAPI didn’t invent it, but is currently the most successful and popular web framework offering it. With the increasing popularity of async, the need for async libraries grew over time. Python’s standard library is sync only for historic reasons. So async packages filled the gap and sometimes…
Problem LocalForage is a thin wrapper around IndexedDB (and some fallbacks). IndexedDB is a NoSQL database API exposed by modern browsers. It allows storing JSON and blobs client-side. LocalForage simplifies interacting with IndexedDB by providing a simple key-value store as API instead. The method names are localStorage -like: getItem() , setItem() , removeItem() etc. However, batch operations…
Aquasec Trivy is an open-source vulnerability scanner for dependencies. It can be used to get a quick overview of all (tagged) Docker images on a host and their respective vulnerabilities. This can be done with the following Compose setup: services : secscan-reports : image : caddy:alpine command : caddy file-server --root /reports --browse --listen :8080 volumes : - ./reports:/reports:ro ports :…
Minimalism https://milligram.io/ https://minicss.us/docs.htm https://picnicss.com/documentation Retro https://nostalgic-css.github.io/NES.css/ https://jdan.github.io/98.css/ https://botoxparty.github.io/XP.css/ https://khang-nd.github.io/7.css/ Components as CSS classes https://www.getpapercss.com/docs/ https://getbootstrap.com/docs/5.2/getting-started/introduction/…
Using 7zip files as encrypted backups is a quick and dirty workaround for a proper backup solution. First generate a strong password, e.g.: pwgen -1sB 128 Backup without compression 7z a -mx=0 -mhe=on -m0=Copy -p"$PASSWORD" TARGET.7z SOURCE_FOLDERS_* mx=0 and m0=Copy mean: just copy the files, do not compress them mhe=on means: also encrypt the 7z headers, so the password is also required for…
Wayland has the safety feature of clearing the clipboard if you close the source window (where the content was copied from). Password managers (like KeePass) also give you only a few seconds to paste the password somewhere before clearing the clipboard. But in day-to-day life, you might need to copy secrets and tokens outside a password manager. To prevent accidental pastes, these values should…
The TICK stack v1 did not enforce authentication. So SSO for the Chronograf WebUI could be handled easily via the reverse proxy. InfluxDB v2 includes a WebUI on its own and therefore replaces Chronograf. As InfluxDB v2 enforces built-in authentication there must be accounts. But InfluxDB v2 has no SSO capabilities. The solution is to create an “All Access API Token” and inject it via…
Create a gitlab user “renovate.bot”: External User: Yes Personal projects limit: 0 Can create top level groups: No Impersonate “renovate.bot” (or login as this user) and create a Settings → Access Tokens : Expiration: 1 year (place a reminder to renew this!) Scopes: api, read_api, read_repository, write_repository, read_registry Create a repository…
The problem In programming there is often a need to run things in parallel, for example: Execution time is optimized if tasks run in parallel A webserver needs to handle multiple requests simultaneously Sending an email in the background should not block the UI There are three different approaches for these requirements with structural differences. Multiprocessing / Multiprogramming…
You must be admin to get a list of all repositories You need a personal access token: Gitlab WebUI → Preferences → Access Tokens → Select scopes: api or read_api → Create personal access token Run the script: import json import requests import csv token = 'YOUR_PERSONAL_ACCESS_TOKEN' page = 1 repos = [] users = {} while True : print ( 'Page:' , page) res = requests . get( f…
Counting rows in postgresql is as easy as select count(1) from mytable . This is a precise count. But it becomes slow on an increasing number of records, especially when a sequential scan of all table records is required, see explain select count(1) from mytable . But in the end the user often doesn’t care if there are about 125_000_000 or exactly 124_756_849 results. Therefore, a fast row…
This post will briefly explain the basic components of a geographic information system (GIS) from a technical point of view (for comp sci students). Part 1 GIS : provides the required tools to perform operations on geospatial data, e.g. QGIS, OpenLayers, Google Maps Map : representation of geospatial information, shows multiple layers which are stacked over each other in a visual manner Projection…
M$ Office is tightly bundled with Internet Explorer. So it used to open links in office documents in a separate IE8 instance. That was already problematic as cookies were not preserved. So the user has to login every time he clicks on a link in a excel sheet if the website has authentication in place. Nowadays the Internet Explorer is finally dead (at least I hope so). When you click on a link in…
Miniflux is a feed reader. As adding feeds is easy, some bloat will appear. Article Age List the top 10 feeds with the oldest articles. No updates, no need to keep a subscription. select f.feed_url, f.title, ( select max (published_at) from public .entries where feed_id = f.id) last_published_at from feeds f where f.user_id = ( select id from users where username = 'knrdl' ) order by…
DNS provisioning mostly works out of the box. But once for a cloud instance the DNS resolver had terrible response times. So I added another resolver. Ubuntu uses netplan as abstraction layer for network config. sudo nano /etc/netplan/01-netcfg.yaml ... nameservers : search : [ invalid ] addresses : - 127.0.0.1 # the new resolver - 123.123.123.1 # provided dns resolver - 223.123.123.2 # provided…
Install docker Comes with Docker Compose as $ docker compose sudo apt-get update sudo apt-get install ca-certificates curl gnupg lsb-release curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg echo 'deb [arch= $( dpkg --print-architecture ) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ $( lsb_release -cs…
the problem Connection Pooling is an often overlooked concept when it comes to databases. An inexperienced developer might implement an API backend like this: HTTP Request comes in Open connection to the database Execute SQL command and read response Close connection to the database (also often forgotten) Send HTTP response Opening and closing a connection (step 2 & 4) produces load on the…
Problem SD (or MicroSD) Cards are cheap flash storage. Their lifetime is limited by the performable write operations. If their end of life is reached, they don’t work at all or read data unreliably. A Raspberry Pi produces more IO operations than a typical digital camera (which SD Cards are conceptualized for). The load is mostly produced by writing: Temporary files in /tmp/ Variable files…
To realize encrypted data volumes, there are various tools for different use cases: Veracrypt works cross-platform, predefined volume sizes Gocryptfs encrypts transparent, dynamic storage usage Restic encrypts backups, network capable, can do versioning LUKS provides full disk encryption for linux systems But LUKS can also be used for regular data volumes which behave a lot like Veracrypt volumes.…
Approach Wireguard is a modern VPN protocol. Per default all nodes are treated equally, named peers. Let’s build a point-to-point connection: Client: Wireguard running, no port opened Server: Wireguard running, one port opened Wireguard uses UDP packets only. If an incoming request is invalid (wrong encryption key) then it will be dropped silently. Thereby additional security measures like…
Concept A docker-registry stores docker-images, composed of: metadata : image names and tags blobs : actual image contents The most famous public docker-registry is DockerHub . DockerHub applies a rate-limiting for downloading blobs. The fetching of metadata is not sanctioned. Therefore, a local docker-registry mirror can be used to circumvent DockerHub’s rate-limiting. This might also…
Scenario Some web hosters only serve static files and allow no config changes to the webserver. But maybe you want to provide files which are not intended for public view, for example sharing a file with a friend. Therefore, the best you can do is protecting files by giving them names which are hard to guess. Obviously these files should also not be linked somewhere publicly at all. This concept…
The HTML input element is the principal construct when it comes to site local interactivity from user provided data on the web. Here are my 5 cents to the topic. Button Tag There is a <input type="button"> . However, it only allows unformatted text as button content. Use <button type="button"> instead. The type="button" might seems redundant, but it prevents browsers from interpreting the button…
Introduction There are three kinds of private ip ranges: Class CIDR Last IP IPs Typical Usage A 10.0.0.0/8 10.255.255.255 16,777,216 Big Company Network B 172.16.0.0/12 172.31.255.255 1,048,576 Docker Network! C 192.168.0.0/16 192.168.255.255 65,536 Home Network So there are about a million IPs available for docker containers in docker networks. However, docker per default splits them up as /24…
PDF files are still highly popular. The user will find all information in a single, human-readable file which is quite durable. However, the programmatic generation of pdf files can be a bit tricky. A rendering engine is needed to transform the source data into the target pdf file. The source data can be broken up into two parts: templates and dynamic data. The template is a skeleton of the pdf…
Each docker image has a URI-style name like domain.tld:port/directory/subdir:tag . The image name ubuntu is just a shortcut for docker.io/library/ubuntu:latest . The default tag (if omitted) is latest . Docker (or OCI ) images should be built automatically in CI -Pipelines. There are multiple strategies for tagging those images: Docker Image Tagging via Git Commit Tags: Relevant git commits are…
The Principle of Least Privilege (PoLP) should also be applied to applications inside a container! Running stuff as root user is never a good idea. A custom user profile with fewer privileges should be created and used instead. But such a user cannot bind a privileged port (<1024), e.g. start a webserver on port 80. Exposure via port binding to the host is still unproblematic, e.g. docker run -p…
In postgres there is nothing like create table if not exists for enums. Workaround: DO $$ BEGIN CREATE TYPE request_type AS ENUM ( 'request_type1' , 'request_type2' ); EXCEPTION WHEN duplicate_object THEN null ; END $$;
Useful when webpage contains password inputs < script lang = 'ts' > let warnCapsLockOn: boolean = false function checkCapsLock(e: KeyboardEvent ) { warnCapsLockOn = !! e.getModifierState( 'CapsLock' ) } </ script > < svelte:body on:keyup = {checkCapsLock} / > { #if warnCapsLockOn} < div class = 'text-danger' >Attention, caps lock on!</ div > { /if }
A notebook as mobile device may should behave differently depending on the location. Examples: at home: sound output should be turned on a backup script can run from time to time in a foreign network / without connection: speakers should be muted a vpn connection should be established in public networks at work: speakers should be muted time tracking software should be active Evaluating the ssid…
The ESP8266/ESP32/NodeMCU boards firmware has builtin support to run a http server. It’s possible to build a restful json api but also to serve static files. Setup Arduino IDE → File → Preferences → Additional Board Manager URLs: http://arduino.esp8266.com/stable/package_esp8266com_index.json Connect to Wifi #include <ESP8266WiFi.h> #include <WiFiClient.h> const char * ssid =…
SSH allows tcp port forwarding between ssh client and ssh server as part of the encrypted ssh connection. Server → Client (local) A socket on the server (source) gets forwarded to the client (target). ssh user@host -L target_ip:target_port:source_ip:source_port Use case 1: Admin software Server socket with port 9000 will be available on client machine at port 5000. ssh user@host -L…
GlusterFs is a distributed storage layer based on the rsync algorithm . GlusterFs works great for semi-static files, but not for databases like sqlite or postgres! Furthermore, inotify-based programs (file change monitoring) won’t work reliable. Setup For best performance the traffic between nodes stays unencrypted. Such a glusterfs should only be operated in a dedicated, trustworthy…
1. find db byte sizes SELECT datname, pg_size_pretty(pg_database_size(datname)) FROM pg_database ORDER by pg_database_size(datname) DESC ; 2. find tables + indices sizes select table_name , pg_size_pretty(pg_total_relation_size(quote_ident( table_name ))) from information_schema.tables where table_schema = 'public' order by pg_total_relation_size(quote_ident( table_name )) desc ; 3. recreate index…
If docker swarm rejects to deploy a service because network interface already exists: $ docker service ps stackname_appname --no-trunc Rejected 34 seconds ago 'network sandbox join failed: subnet sandbox join failed for ' 10.0.14.0/24 ': error creating vxlan interface: file exists Then find all problematic interfaces on the host and delete them: $ ip -d link show | grep vx | grep DOWN $ sudo ip…
Docker (or podman) to the rescue! Each first line shown is the Docker command. And the second line is the corresponding command for Podman. Setup project sudo docker run --pull always -it --rm --user ' $UID : $UID ' -v ' $PWD : $PWD ' -w ' $PWD ' node:alpine npm init podman run --pull always -it --rm -v ' $PWD : $PWD ' -w ' $PWD ' node:alpine npm init Development Svelte has replaced rollup with…
The gitlab docker registry has no cleanup job per default. If an image tag gets overwritten (updated) then the original image layer blobs will be kept as orphans. To clean those up run: gitlab-ctl registry-garbage-collect --delete-untagged --delete-manifest Automation run the command via Cron (the gitlab image contains go-crond ) set command in env var GITLAB_POST_RECONFIGURE_SCRIPT (executed at…