v1.0 released · Apache 2.0 · open source · PostgreSQL 15–18

pg_hardstorage is an open-source PostgreSQL backup tool — built so something as fundamental as your recovery story stays code you can read, run, and rely on. WAL streaming over the replication protocol, content-addressed storage where every backup restores on its own, and a design that values boring, verifiable reliability over cleverness. Originally developed by CYBERTEC, now a community project.

0 Backup Chains to Break

AES-256 Encryption On by Default

Apache 2.0 Open-Source Licence

"Backup is the one piece of infrastructure where 'trust us, it's proprietary' is the wrong answer. pg_hardstorage is engineered to be inspected — by you, by your auditors, by anyone."

Data plane

WAL via the replication protocol

The agent connects to PostgreSQL the same way a streaming replica does — a libpq connection, nothing else. No SSH. No archive_command. No host access. That is why a bare-metal server, a Patroni cluster, and PostgreSQL behind a Kubernetes operator all take the same setup: there's no special privilege, just a replication-role user that can run BASE_BACKUP.

Repository

Content-addressed dedup

FastCDC chunker with page-aligned splits. Every chunk keyed by its plaintext SHA-256, so the chunk store itself has no chains — a chunk is shared by hash, never copied. A repeat backup deduplicates against the deployment's existing chunks, so unchanged data costs almost nothing in storage. (PG 17 incremental backups still record their parent, and retention keeps that parent pinned.)

Encryption

AES-256-GCM envelope, FIPS-ready

A per-backup data key (DEK) encrypts every chunk with AES-256-GCM under a unique random nonce; the DEK is wrapped by your KMS. AES-256-GCM by default; a BoringCrypto FIPS build variant is available. Crypto-shred by destroying the wrapping key — Article 17 compliance in one command.

Pluggable

KMS & storage backends, pluggable

Keys: AWS KMS, GCP KMS, Azure Key Vault, HashiCorp Vault Transit, and a local keyring (a PKCS#11 / HSM plugin is in progress). Storage: filesystem, S3, GCS, Azure Blob, SFTP and SCP. Tier-2 plugins ride a stable gRPC-over-stdio protocol — write your own in any language that speaks gRPC.

High availability

Patroni-aware failover

Four cooperating mechanisms — Patroni permanent_slots, PG 17 synced slots, recreate-on-detection, and the gap auditor — keep WAL streaming through leader switches.

Recovery

Point-in-time, in plain English

--to "5 minutes ago", --to-lsn 0/3000028, --to-name <name>. The orchestrator picks the latest full whose stop-LSN precedes the target, replays WAL, applies timeline switches, and prints a projected RTO before it starts.

Verification

Sandboxed full-restore verify

Two verifier sandboxes — Docker by default, Firecracker microVM with the -tags firecracker build. Restore into a throwaway PG and run pg_verifybackup, then tear it down. A backup you didn't restore is a backup you don't have.

3am operator surface

LLM-assisted, not LLM-controlled

Optional. Read-only by default. Every suggestion footnoted to its source tool call, every session hash-chained into the audit log. Advise+execute is opt-in and gated by a five-stage safety stack. The 3am operator gets help — the 3am operator does not get an autonomous agent with shell on the data directory.

Output

Structured everywhere

Every command supports --output json or --output ndjson with the pg_hardstorage.v1 schema. 14 sinks · 11 renderers · OpenTelemetry traces · Prometheus metrics on pg_hardstorage_*. Wire format committed to 24-month back-compat.

Most native

Kubernetes

PostgreSQL behind a Kubernetes operator is standard Postgres in a pod — pg_hardstorage backs it up over the replication protocol like any instance you run yourself. Verified end-to-end against CloudNativePG: backup, verify, and restore all round-trip cleanly. Run it as a CronJob or a Deployment.

• CloudNativePG — backup/verify/restore verified
• Runs as a CronJob or a Deployment
• Connects over the standard replication protocol
• In-tree Helm chart; no operator plugin yet

Bare-metal Linux

RPM and .deb packaging for RHEL, Rocky, AlmaLinux, Fedora (≥ 9), Debian, Ubuntu. systemd units in deploy/systemd/; a one-liner ./compile.sh for custom builds.

• RPM / .deb / systemd units
• FHS / XDG paths respected
• Distroless container variant

Containers

The repository ships deploy/docker/Dockerfile — a small static image you build and host yourself. The release pipeline (see .goreleaser.yaml) cosign-signs artefacts and emits an SPDX SBOM.

• Docker / Podman / containerd
• Static binary · runs as nonroot
• Cosign-signed releases · SBOM

Air-gapped

Repo bundle export/import for moving backups across security boundaries. Air-gap policy refuses any plugin that performs network I/O. Runbook-driven cold-start from offline media.

• Repo bundle export / import
• Network-deny policy mode
• Offline KMS support

FIPS / Compliance

A BoringCrypto FIPS build variant (make build-fips). Releases are cosign-signed with an SPDX SBOM. The hash-chained audit log can be anchored to a transparency log with audit anchor.

• FIPS build variant
• Cosign-signed releases · SBOM
• Hash-chained, anchorable audit log

VS

Apache 2.0 Open-source licence you can read and fork

Open Public roadmap, issues, and development

Zero Lock-in — your repo, your format

Maintainer's promise

The bar is "this would land in the project anyway." Tight, technical PR descriptions get tight, technical reviews. No CLA, no rights assignment — your DCO sign-off and Apache 2.0 is enough. Maintained by Hans-Jürgen Schönig at CYBERTEC PostgreSQL International, but owned by everyone who ships a patch.

DCO Not a CLA

Apache 2.0 Every line

Public Roadmap & issues

Daily operations cheat-sheet

• pg_hardstorage backup db1 --pg-connection … --repo …Take a backup right now
• pg_hardstorage restore db1 latest --repo … --target … --to "5 minutes ago"Natural-language PITR
• pg_hardstorage status db1 --repo …Deployment status
• pg_hardstorage verify db1 latest --repo …Verify chunk hashes + manifest signature
• pg_hardstorage doctor db1Connection + repository health
• pg_hardstorage rotate db1 --repo … --keep-for 336h --applyRetention (dry-run without --apply)

Read the full documentation →

No. pg_hardstorage takes its base backup with the BASE_BACKUP replication-protocol command. Managed database services do not expose that command to customers — they do not let an external client run a physical base backup or attach a physical replica. So pg_hardstorage cannot back those services up, and we do not claim it can. It targets PostgreSQL you operate yourself: bare-metal and VM installs, Patroni clusters, and PostgreSQL behind a Kubernetes operator such as CloudNativePG.

PG 15, 16, and 17 are first-class. PG 18 is in the test matrix. PG 14 and earlier are out of scope — BASE_BACKUP non-exclusive mode and pg_backup_start / pg_backup_stop are PG 15-only APIs we rely on.

The pg_hardstorage open-source project is licensed under Apache 2.0. Compliance reporting, encryption, the audit chain, the FIPS build variant, and the LLM helper are all part of the project. It was originally developed by CYBERTEC PostgreSQL International GmbH and is now maintained with the community.

The short version: WAL via the replication protocol (works on managed PG), no chained incrementals (CAS dedup with no chain dependency), encryption / KMS / audit chain / WORM on by default. pgBackRest's strengths — production maturity at very large scale, mature operator integrations — are real, and we're explicit about the cases where each tool wins. Side-by-side migration paths exist for all three.

Yes. pg_hardstorage restore <deployment> --to "yesterday 9pm", or any other natural-language time, or --to-lsn 0/3000028, or --to-name <name>. The orchestrator picks the latest full whose stop_lsn precedes the target, replays WAL, and applies any timeline switches. --preview dry-runs the plan first.

The chunk store is content-addressed: every backup's manifest references the chunks it needs directly by SHA-256, and a chunk is shared, never copied. PG 17 incremental backups do record a parent backup, and retention pins that parent so it can't be pruned out from under its incrementals — but full backups stand alone, and there is no fragile differential chain to replay. More in the architecture page.

Restore is bandwidth-bound. Today the chunk fetch is single-stream, so end-to-end time tracks your link and storage throughput plus pg_verifybackup once the data dir is hydrated; parallel fetch is a planned optimisation. --preview prints a projected RTO (from a conservative throughput estimate) before kicking off.

Wherever you point the encryption plugin: AWS KMS, GCP KMS, Azure Key Vault, or HashiCorp Vault Transit (a PKCS#11 / HSM plugin is in progress). For single-host use, a local aes-256-gcm keyring directory works. KEK rotation re-wraps existing backups' DEKs without rewriting any chunks.

Yes. The audit log is hash-chained Merkle, periodically anchored to a transparency log (Rekor by default, customer-managed log on request). compliance report generates JSON + Markdown reports that map observed events to control IDs (PDF rendering is on the roadmap). Signed evidence bundles are produced via audit export-bundle.

Apache 2.0 means anyone can fork. The wire formats are documented and schema-versioned with a 24-month backward-compatibility commitment, so any fork can read your existing repository. Manifests are plain JSON; chunks are content-addressed. Your data is recoverable without the original vendor — by design.

Use the GitHub Security Advisories tab on the repository — an advisory is a private draft until disclosure. Don't open public issues for security findings. The disclosure policy is in SECURITY.md.

In-tree under docs/, and published at docs.pghardstorage.org. The site is built with MkDocs Material (make docs-build). The information architecture is Diátaxis: every page is either a tutorial, how-to, reference, or explanation. Plus seven named runbooks (R1–R7) for the disaster scenarios you'd otherwise discover at 3am.