I run Blocky as my network-wide DNS resolver. It sits on a Dell OptiPlex i5-9500T with 16GB of RAM, handles every DNS query for every device in my house, and blocks ads for the whole network. Phones, laptops, TVs, a Tesla, an Xbox, Kubernetes nodes — everything flows through this little box.
I’ve been running Blocky for a while now, but recently I went down a rabbit hole of squeezing every last millisecond out of it. This post documents everything I tried, what worked, what didn’t, and the final numbers.
Spoiler: I got cold cache DNS resolution down to 4.9ms average with a p99 of 14ms. On a cheapo slim desktop (that’s sitting on the carpet under my desk in my office).
Blocky v0.29.0, running bare-metal with systemd. No Docker, no containers. The config was pretty standard — single upstream (Cloudflare 1.1.1.1), basic caching, default everything. Cold cache latency was hovering around 7-8ms average, which is fine for most people. I know I have a problem. Literally EVERYONE should be satisfied with this. However, I knew I needed it lower.
This was the single biggest win. Instead of querying one upstream DNS resolver and waiting, parallel_best fires queries to all your upstreams simultaneously and takes the fastest response. More upstreams = more chances of getting a fast answer.
I tested a bunch of combinations. The winner: 2 Cloudflare + 2 Quad9.
upstreams:
groups:
default:
- 1.1.1.1
- 1.0.0.1
- 9.9.9.11
- 149.112.112.11
strategy: parallel_best
timeout: 200msThat timeout: 200ms is aggressive. Default is 2 seconds, and I had it at 500ms for a while. But with 4 upstreams racing, you only need ONE to respond fast. If an upstream is having a bad day, cut it off at 200ms and the others will carry. This alone chopped the p99 from 80ms to 14ms.
I benchmarked every upstream from the server to find the fastest ones:
UpstreamAvg latency from server1.1.1.1 (Cloudflare)4ms1.0.0.1 (Cloudflare)4ms9.9.9.11 (Quad9)4ms149.112.112.11 (Quad9)4ms172.64.36.1 (Cloudflare alt)13ms172.64.36.2 (Cloudflare alt)15ms45.90.28.59 (NextDNS)3-4ms45.90.30.59 (NextDNS)3-4ms
The Cloudflare 172.64.36.x IPs look like they’d be the same infrastructure but they route to a completely different anycast POP from my network. 3x slower. Geography matters.
NextDNS looked promising at 3-4ms per query, but when I actually ran it through Blocky the p99 was 101ms and max was 195ms. NextDNS does server-side filtering and logging on every query, so that processing overhead shows up in the tail. With only 2 upstreams, there’s no third racer to save you when both hit a slow moment.
I also tried Cloudflare’s malware-blocking endpoints (1.1.1.2 / 1.0.0.2). Removing them actually improved performance — they were marginally slower and were dragging parallel_best‘s scheduling.
Blocky is written in Go. Go’s garbage collector runs periodically and pauses the program briefly. By default it triggers when the heap doubles (GOGC=100). Setting GOGC=400 means GC runs 4x less often, at the cost of using more memory. On a box with 16GB of RAM running a single DNS service, that’s a great trade.
The flip side of GOGC=400 is that when GC does run, it has more work to do and the pause can spike. GOMEMLIMIT (Go 1.19+) gives the runtime a soft memory target. Instead of one big GC spike, it smooths the work out over time. Set it in the systemd unit:
[Service]
Environment=GOGC=400
Environment=GOMEMLIMIT=2GiBcaching:
minTime: 4h
maxTime: 72h
maxItemsCount: 500000
cacheTimeNegative: 5m
prefetching: true
prefetchExpires: 24h
prefetchThreshold: 1
prefetchMaxItemsCount: 100000The key settings here:
prefetchThreshold: 1— after a domain has been queried just once, Blocky will proactively re-resolve it before the cache entry expires. This means popular domains are almost always warm.prefetchExpires: 24h— domains queried in the last 24 hours keep getting prefetched. I started at 12h and bumped it up.cacheTimeNegative: 5m— negative results (NXDOMAIN) get cached for 5 minutes instead of default 30 minutes. If a domain comes back to life, you’ll know sooner.maxItemsCount: 500000— 500K cache entries. RAM is cheap.
queryLog:
type: none
ede:
enable: false
filtering:
queryTypes:
- AAAAQuery logging writes to disk on every query. Disabled. Extended DNS Errors (EDE) adds processing overhead per response. Disabled. AAAA filtering drops IPv6 lookups since I’m on an IPv4 network — cuts upstream query count in half for dual-stack domains.
This is where it gets fun. Everything below goes in /etc/sysctl.d/99-dns-perf.conf:
# UDP buffer minimums for DNS burst handling
net.ipv4.udp_rmem_min = 65536
net.ipv4.udp_wmem_min = 65536
net.core.optmem_max = 131072
# Keep kernel caches hot
vm.vfs_cache_pressure = 50
# Busy-polling — poll for packets instead of sleeping
net.core.busy_poll = 50
net.core.busy_read = 50
# Prevent paging latency spikes
vm.swappiness = 1
# NAPI polling budget — process more packets per cycle
net.core.netdev_budget = 600
net.core.netdev_budget_usecs = 4000
net.core.dev_weight = 128
# Simpler qdisc — avoids fq_codel spinlock contention on UDP
net.core.default_qdisc = pfifo_fastLet me break these down.
Busy-polling (busy_poll / busy_read) tells the kernel to actively poll for incoming packets for 50 microseconds instead of sleeping and waiting for an interrupt. Costs a tiny bit of CPU but shaves latency off every packet.
NAPI budget controls how many packets the kernel processes per polling cycle before yielding to other work. Default is 300 packets / 2000us. I bumped it to 600 / 4000us since a DNS server handles bursts of small UDP packets and I want them all processed in one shot.
Qdisc: pfifo_fast over fq_codel. fq_codel is the default and it’s great for general-purpose networking. But it adds spinlock contention and flow-tracking overhead that a DNS server doesn’t need. pfifo_fast is simpler and lower latency for UDP.
Transparent Huge Pages (2MB) are notorious for causing latency spikes in Go programs due to memory compaction stalls. I disabled those. But kernel 6.8 introduced multi-size THP (mTHP) — you can enable 64kB pages without the 2MB compaction overhead. This gives Go’s heap fewer page faults without the downsides.
# Disable 2MB THP (compaction stalls)
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/hugepages-2048kB/enabled
# Enable 64kB mTHP (kernel 6.8+)
echo always > /sys/kernel/mm/transparent_hugepage/hugepages-64kB/enabledGeneric Receive Offload lets the NIC aggregate small packets before handing them to the kernel. One-liner:
ethtool -K eno1 gro onEvery packet on a Linux box goes through the connection tracking subsystem (conntrack) by default. For a DNS server that doesn’t do NAT, this is pure overhead and adds jitter under load. I bypass it entirely for port 53:
iptables -t raw -A PREROUTING -p udp --dport 53 -j NOTRACK
iptables -t raw -A PREROUTING -p udp --sport 53 -j NOTRACK
iptables -t raw -A OUTPUT -p udp --dport 53 -j NOTRACK
iptables -t raw -A OUTPUT -p udp --sport 53 -j NOTRACKSame for TCP. Persist it with a systemd oneshot service so it survives reboots.
# CPU governor to performance (max clock, no frequency scaling)
echo performance > /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
# Real-time scheduling in systemd unit
CPUSchedulingPolicy=rr
CPUSchedulingPriority=50RT scheduling gives Blocky priority over background tasks on the CPU. On kernel 6.8, the old RT throttling mechanism was replaced with “deadline servers” which makes this smoother and more deterministic — no more periodic RT starvation.
Not everything I tried was a win. This is the section I wish more blog posts included.
Tried pinning Blocky to specific CPU cores (2 cores, then 4 cores). Made it worse. Go’s goroutine scheduler is designed to spread work across all available cores, and restricting it causes scheduling contention. p99 jumped from 1ms to 13ms. Reverted immediately.
Blocky supports Redis for shared caching across multiple instances. For a single home server, it’s pointless — the in-memory cache rebuilds in seconds after a restart, and adding a network hop to Redis for every cache lookup defeats the purpose.
I tried adding seccomp syscall filtering, RestrictNamespaces, MemoryDenyWriteExecute, SystemCallFilter=@system-service, etc. Got the systemd security score from 5.6 to 2.4. Looked great on paper. Then I benchmarked it.
Added ~0.5ms to cold cache average (6.5ms → 7.0ms). Every Go syscall was going through seccomp BPF filtering. For a home DNS server with minimal attack surface, the security benefit wasn’t worth the latency cost. Dropped it.
NextDNS (45.90.28.59 / 45.90.30.59) looked competitive in raw latency tests (3-4ms). But through Blocky with parallel_best, the p99 was 101ms and max was 195ms. NextDNS does server-side filtering/logging, and with only 2 upstreams racing there’s no safety net for slow responses. Cloudflare + Quad9 with 4 upstreams was consistently faster.
These are Cloudflare’s “for families” endpoints that block malware domains. When added to the upstream pool, they were marginally slower than the standard 1.1.1.1 endpoints. Removing them actually improved performance — cold avg went from 5.9ms to 4.9ms. Fewer but faster upstreams won.
When I upgraded to kernel 6.8 (Ubuntu 24.04), several things got faster for free:
Cacheline-optimized networking structs — Google reworked
struct sockandstruct netdeviceto minimize cache misses. Every socket operation benefits.EEVDF O(1) scheduler — replaced CFS with an O(1) fast path for task selection. Lower scheduling overhead, less jitter.
Deadline servers for RT scheduling — smoother RT behavior without the periodic throttling starvation of the old mechanism.
SLUB allocator optimizations — “delayed freezing of CPU partial slabs” yielded a 34% improvement in memory allocation microbenchmarks.
None of these required configuration. Just run the new kernel.
I wrote a benchmark script that tests 50 domains across 3 iterations for both cold cache (restart Blocky, query with empty cache) and warm cache (query again from cache). It also tests Numa (a Rust-based DNS resolver) for comparison.
Here’s where I ended up:
Blocky
Cold cache avg: 4.9ms
Cold cache p95: 8ms
Cold cache p99: 14ms
Cold cache max: 16ms
Warm cache avg: 0.3ms
Warm cache p99: 1ms
Numa (Rust-based, for comparison)
Cold cache avg: 6.6ms
Cold cache p95: 20ms
Cold cache p99: 80ms
Cold cache max: 85ms
Warm cache avg: 0.2ms
Warm cache p99: 1ms
For context, here’s the journey:
Starting point (stock config) — cold avg ~8ms, p99 ~100ms+
After app-level tuning — cold avg 7.7ms, p99 ~80ms
After system-level tuning — cold avg 5.3ms, p99 ~17ms
After kernel 6.8 + tail-latency tuning — cold avg 4.9ms, p99 14ms
Everything is codified in a GitHub repo: bootstrap script, config, benchmark script, swap script (for toggling between Blocky and Numa). Run the bootstrap on a fresh Ubuntu box and it handles everything — binary download with SHA256 verification, systemd unit, sysctl tuning, THP, GRO, conntrack bypass, CPU governor, the works.
The biggest lessons from this exercise:
More upstreams with parallel_best is the highest-leverage change. It’s free and it crushes tail latency. 4 upstreams racing means one slow resolver can’t ruin your day.
Aggressive timeouts matter more than fast averages. Dropping the upstream timeout from 500ms to 200ms didn’t help the average much, but it obliterated the p99. Chase the tail, not the mean.
Always benchmark before and after. Intuition is wrong half the time. CPU pinning “should” help a DNS server. It didn’t. Systemd hardening “shouldn’t” affect latency. It did. Numbers don’t lie.
System-level tuning adds up. No single sysctl setting was transformative, but the combination of busy-polling + NAPI budget + qdisc + conntrack bypass + mTHP + GRO shaved real milliseconds.
Know when to stop. I’m at 4.9ms cold cache average. The upstreams themselves take 3-4ms. I’m within ~1ms of the physical limit. Diminishing returns are real, and the perfect setup does not exi—
Cheers,
Joe
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.