Honest daily notes on building Pinchy, a self-hosted enterprise AI agent platform built on OpenClaw.
Four days after 0.9.0, a bugfix release that exists because of one finding: on an instance that had locked its domain — the hardening we tell people to do — Pinchy's own plugins were being turned away by Pinchy's own security check. The knowledge base returned nothing on every question, Telegram conversations were never written to Pinchy's store, and an integration with expired credentials could…
Our release checklist used to say the candidate had been clicked through on staging. Clicking through confirms the app boots. So instead I stood up a real knowledge-base agent on the real staging instance: a separate Ollama, an embedding model, a folder of documents, an index run, and then actual questions. Three findings, and none of them could have come from a test suite. The setup guide…
A patch release is cut from the release branch, not from main, and on the day I started assembling 0.9.1 main was already 424 commits past the 0.9.0 tag. Thirty of them belonged in a patch — the rule being that a fix belongs only if the bug actually reaches someone running 0.9.0, which is much narrower than 'it is a fix commit'. What I want to remember from the day is not the backport but the two…
The day after a release is for the website, and the website was showing product screenshots from June 18th. Not because anything failed: the capture workflow ran on every release, uploaded its artifact, and reported success, and the website workflow dutifully downloaded the newest artifact it could find and deployed it. The problem is that the capture workflow is invoked by the release workflow,…
Three weeks after 0.8.0, the release that gives agents your documents. Knowledge Base agents index the folders you mount, search them with vector similarity and Postgres full-text together, and answer with citations down to the page — and the embedding runs on the Ollama you already run yourself, so no document and no vector ever reaches a hosted API. Also in: IMAP mailboxes alongside Gmail and…
Pinchy's 'Test connection' button for a mailbox connected to any host and port an admin typed, and reported precisely why the attempt failed: refused, timed out, TLS error, credentials rejected. Those distinctions are the entire point of a diagnostic. They also turn the endpoint into an internal port scanner with a friendly UI, one request at a time, reaching loopback services inside the container…
Every local install of Pinchy told its owner 'Your Pinchy instance is not secured. Lock your domain to enable HTTPS hardening.' On http://localhost:7777 that advice cannot be taken — there is no domain to lock, and the browser already treats localhost as a secure context precisely because the traffic never leaves the machine. The banner wasn't noisy, it was wrong. Fixing it meant refusing the easy…
A production agent looked stuck: one model turn took 543 seconds and the audit trail said nothing. It had not crashed. The session had grown to roughly 633,000 input tokens and had compacted exactly zero times, because compaction fires at the context window minus 16K, and for a model advertising a 999,424-token window that threshold sits at 983,040, which no real session reaches. The model with…
No pull requests merged, no commits, no work at all. A deliberate day off, which for me is a newer habit than it should be. The reason this gets a post rather than a gap in the numbering is that the numbering counts days, not output, and a devlog that only appears on productive days is a filtered record. I spend most weeks arguing that filtered records are a form of lying: a benchmark that quietly…
Two things stopped being hardcoded today, and they had the same shape. The reliability benchmark could only grade invoice tasks, because every part of it that reads back a result named account.move somewhere (PR #871). And Pinchy could only talk to five named providers, because the provider name was a fixed union in the type system (PR #905). Generalizing the benchmark had to prove it moved no…
Agents could read data from Odoo, from a mailbox, from the knowledge base, and then had no way to give you any of it as a file. Today they do: a single tool renders tabular data an agent already holds into CSV, XLSX or PDF and hands it back as a download in chat (PR #893), riding the delivery path that shipped two days ago. The alternative was an exporter per integration, which looks reasonable at…
Release-testing the IMAP connection end to end on staging, I opened the audit row for a sent email and found the recipient address and the complete message body sitting in it verbatim. The tool-use route logs raw tool parameters unless a tool returns curated details, and the email tools never curated (PR #904). That trail is append-only and HMAC-signed, which is the entire point of it, and which…
A customer upgraded their self-hosted box the way people actually upgrade: pull the new images, keep the compose file you already have. That compose file predated a volume mount, so Pinchy could no longer write its secrets file, config regeneration aborted mid-flight, and openclaw.json froze at its pre-upgrade content. Every provider they added and every agent they created after that saved to the…
The reliability index could tell you which models stay honest when a tool fails, but not what a task costs to attempt, which is half of any real decision. Today it captures per-run token cost, joined exactly to each run through a unique session key (PR #839). The measurement decision that matters is the median: cost is reported over completed runs only, because a model that gives up fast burns few…
Connect a mailbox to Pinchy on a cloud server and the test could fail with a bare 'Connection timed out,' which reads as 'your mailbox settings are wrong.' Often they aren't. Most cloud hosts block outbound SMTP ports by default, so sending on port 465 times out no matter how correct your credentials are (PR #847). The test now runs the receiving and sending legs separately, and when the sending…
A week ago I shipped GDPR pseudonymization for the audit trail: every row references a per-user pseudonym, so deleting a user makes their history unlinkable. On staging I checked how well it was working. Two rows out of 1221 were actually pseudonymized (PR #845). Every tool event carried the raw user id in the clear. The cause was a case mismatch: OpenClaw lowercases its session keys, the id…
During staging verification for the next release, Pinchy started crash-looping about five seconds after it said 'ready.' The cause (PR #843) was a database join that fanned a large blob across every row that referenced it. One Odoo connection carried an 837 kB cached catalog in its data column, 426 permission rows pointed at that connection, and the unprojected join materialized the blob once per…
Yesterday the reliability benchmark's grader was the thing under scrutiny. Today it's the benchmark's credibility as a published artifact. Benchmarks mostly die of predictable, self-serving failures: contamination once the questions leak into training data, unsolvable tasks nobody checked, hidden funding, and overclaimed separations between models that are statistically tied. So the work was…
Agents couldn't remember things across sessions, so a rule someone taught an agent (which account certain invoices post to) got applied wrong the next day. Yesterday's fix restored reading memory with an offline embedding model. Today's found the other half: they couldn't reliably write it either. Pinchy's config granted each agent write access to a memory folder it had never actually created, so…
Pinchy publishes a reliability index that grades open-weight models on one thing: when a tool fails, do they report the failure or fabricate success? Auditing my own dataset, I found the grader was crediting crashes as honesty (PR #716). Seventeen runs had died on network errors and been scored as passes, because a model that says nothing tells no lie, and one published model narrative was simply…
A bookkeeping agent was sent a PDF ticket and spent two turns insisting the file 'seems not yet fully available in the workspace.' The file was fine: 244,491 bytes, present and identical in both containers. It looked like the agent was failing at something simple. It wasn't. Two independent tool bugs were wearing its face (PR #724, #729, #732). First, the model reached for OpenClaw's built-in pdf…
The first durable piece of the Inbox Agent landed: the ledger that tracks which emails a workflow has already handled (PR #710), built on the background-jobs foundation from the org-timezone slice (PR #707). No poller, no dispatcher, no agent run yet, just the data layer, because there's no caller for the rest. The design decision worth writing down is what answers 'have we already processed this…
Share to Pinchy lets you send a photo, PDF, or link from any Android app straight into an agent chat (PR #708). Getting there meant a login redirect that remembers where you were headed, and that redirect is where review caught a critical open redirect before it merged. The guard checked that the return path started with a slash. The string '/\t/evil.com' passed it. The browser then strips the tab…
Send a receipt to an agent over Telegram and something strange happened: the model could see the image, but no tool could open it. OpenClaw downloads inbound media into its own store, outside every path Pinchy's file tools are allowed to read, so odoo_attach_file found an empty uploads folder and the agent papered over the gap by inventing a filename (PR #696). The fix mirrors each file into the…
The audit log now verifies its own hash chain in the background and can crypto-shred a deleted user without breaking the trail (PR #691). Two details made the day. First, an append-only table protected by row-level triggers still had a hole: TRUNCATE is a statement-level operation in Postgres, so the no-delete trigger never fires for it, and one TRUNCATE could wipe the history the triggers were…
Yesterday's fix broke up a fight between two Pinchy copies after it started. Today's fix (PR #685) tries to catch it at the door: when you connect a Telegram bot token, Pinchy runs a one-second probe first, and if Telegram answers with the real 409 conflict, it refuses the connect before writing any config. The design choice that matters is what the probe does when it isn't sure. It only blocks on…
Telegram lets exactly one process poll a bot token at a time. The loser gets a 409, 'terminated by other getUpdates request.' So when a second Pinchy deployment picked up a token an existing one was already using, each saw the other's 409 and each tried to shut itself down, and the bot went dark everywhere. The fix (PR #686) is a heuristic with a bias: only the recently-added connection backs off,…
Pinchy's balanced default on the Ollama-Cloud stack moved off glm-4.7 to kimi-k2.6 (PR #672, #671), and the reason is a specific, unglamorous incompatibility: glm-4.7 is reasoning-by-default and gets stuck in the reasoning_content round-trip that the /v1 tool-calling path expects, while kimi-k2.6 is a strong tool driver that doesn't insist on thinking out loud first. A 'balanced' default has to…
An admin re-set-up the Microsoft OAuth app, logged in, and got the app config but no integration. Pinchy did everything right, deleted the pending row, audited the real token_exchange_failed, redirected with an error, but the only thing the user saw was a toast that auto-expires in about four seconds, gone before their eyes were even back from the provider's redirect (PR #664). The house style…
Pinchy's email support was Gmail-shaped down to the bone, raw label strings and raw Gmail query syntax leaking straight into the agent's world. Adding Microsoft 365 (PR #328) forced the right refactor first: a single EmailAdapter contract of list, read, search, draft, and send, with canonical folder names (INBOX, SENT, DRAFTS, TRASH, SPAM) and a structured search DSL that neither Gmail nor Graph…
The shape I keep coming back to for a normal company isn't one all-knowing assistant, it's one agent per job: a bookkeeper that lives in the accounting system, a mailbox agent that drafts replies, a support agent that knows the docs. That's a positioning choice with real engineering consequences, because it only works if adding a job is cheap and each agent's boundaries are tight. This week's…
This week's zombie-server incident sharpened a question that self-hosting forces and SaaS lets you dodge: when the software breaks on a customer's own hardware, how does anyone fix it without me in the room? I can't SSH into their box, and I shouldn't be able to. So the diagnostics have to travel to me instead of me traveling to the machine. That's the strategy behind the work this week: a…
A staging container reported perfect health while being completely broken. HTTP was up, /api/health said ok, Docker's healthcheck was green, and every chat session showed 'Reconnecting to the agent…' forever, because the OpenClaw client had thrown at boot and the startup chain had no terminal catch, so the failure surfaced as an unhandled rejection that Next.js logs and swallows (PR #652). A…
A user sends an image to a text-only agent, tabs away, comes back minutes later, and their own message is gone from the thread while the agent's reply remains. The on-disk session was intact the whole time (PR #637): OpenClaw's history RPC caps a single message at 128 KB, an inline image trips it, and the oversized turn is swapped for a placeholder that discards the text and the attachment marker…
A bookkeeping agent on production spent about two hours failing to post a simple opening balance, trying roughly thirty ways to name the accounting journal and getting rejected on every one. The root cause was two bugs braided together (PR #614): the Odoo instance has two companies that each own a journal with the identical name and code, so any lookup by name was ambiguous by construction, and…
Before I cut v0.8.0 I run the same unglamorous ritual: click through the whole app on a staging instance with synthetic data, like a confused first-time user, and write down everything that feels wrong. It never comes back empty. This round it found two bugs (PR #612) that no unit test would have caught by construction: the sidebar reopened an older chat because its store listened only for…
A quiet Sunday to say the strategic thing out loud. The model layer is the fastest-churning part of this whole stack: models get retired with no notice, a 'balanced' default turns out to loop on a reasoning round-trip, open-weight models leapfrog each other monthly. If your product's value lives in a specific model, you're renting your foundation from a landlord who redecorates without telling…
The single most requested capability I haven't shipped is a real browser: agents that can read JavaScript single-page apps and pages behind an anti-bot challenge, which Pinchy's static fetch can't touch. OpenClaw already ships a native Chromium tool, so the hard part isn't building it, it's that I refuse to expose it until the gate around it is real. Yesterday's fail-closed allowlist was the…
Reviewing an unrelated web fix, I found that Pinchy's per-agent tool gating was a deny-list, and a deny-list is only as complete as your memory of everything you have to deny. OpenClaw's native browser and canvas tools live in a group Pinchy never denied (PR #603), so the real browser was reachable by every governed agent, saved from being a live hole only by the accident that the production image…
Production PDF reads on the Ollama-Cloud-only stack started failing, and the audit log named the cause exactly: the vision model Pinchy had pinned was retired by Ollama Cloud on 2026-06-16, so every image and PDF read 410'd until the next upgrade and restart. This was the third time a cloud model vanished under us (devstral-small-2, gemini-2-preview, qwen3-vl), and Ollama gives no advance-notice…
v0.7.0 ships three things that all answer the same question: how does a self-hosted agent platform get more capable without getting less governable. The big one is a skills foundation built on OpenClaw 2026.6.x's native skill mechanics — SKILL.md, per-agent allowlists — with web search as the first pilot skill and a market-monitor template that uses it. The honest framing: it's a foundation and…
The OpenClaw runtime image was shipping its own build toolchain to production: a C compiler, Python, and a 300 MB npm download cache, none of which the gateway uses once it is running. They were there only to compile native modules at build time, then rode along into every deployment. Splitting the image into a builder stage that compiles and a clean runtime stage that copies only the finished…
On Day 120 the chat said the agent didn't respond when it actually had. This is the opposite failure: in production an agent hit a provider rate limit mid-run, right after it had created draft bills in Odoo, and the chat showed nothing at all. The live error bubble was ephemeral client state, so a reload or a websocket reconnect erased it, leaving the failure recorded only in the audit log where…
For months I had careful FAQ answers on 62 marketing pages, and 59 of them showed the answers to nobody: they lived only as FAQPage JSON-LD in the head, fed to a Google rich result and invisible to humans. Then Google dropped FAQ rich results in May 2026, and the schema I was maintaining was working for an audience that had left. The fix is a single Faq component that renders the visible Q&A and…
People ask what Pinchy runs on once you take it fully off the network, and the honest answer is that I have never touched the class of machine they mean. So this is the honest version of a spec sheet: what I can work out about an air-gapped LLM box I have never run, and what I cannot. The arithmetic of a memory-bound machine (bandwidth over TOPS, mixture-of-experts over dense, a 120B model in the…
Prepping an OpenClaw runtime bump, a Telegram test started failing deterministically: the read-only mirror read OpenClaw's session-scoped chat history, which a /new reset empties, so after a reset the web mirror went blank while Telegram itself still showed every message. Reading that storage more cleverly was the wrong fix. The right one is architectural: Pinchy now owns the conversation…
A slash command typed in Telegram could erase the web chat history with the same agent. Not by deleting anything: an accidental /new from Telegram reset an OpenClaw session that, through identity links, was shared with the web UI, so the web transcript silently became unreachable. The fix adopts OpenClaw's per-task session model, keyed per chat, so web and Telegram are separate conversations and…
On production, the chat kept saying 'The agent didn't respond' while the agent had in fact responded: a refresh revealed the saved reply, and anyone who didn't refresh never saw the answer. The cause was five independent heuristics, each on its own clock, all guessing from silence about whether the run was alive. The fix was to stop guessing and use the one component that actually knows: OpenClaw…
I point the same scrutiny at the marketing that I point at the code, and it does not come out clean: a site-wide audit, cross-checked against the shipped product, finds 56 claims that don't match what Pinchy actually does. We advertised channels we don't have, billing we don't do, and an approval gate that's really just an allow-list. The worst one is a security statistic we got wrong on our own…
v0.5.8 shipped today: lossless per-turn token accounting, PDFs rerouted away from OpenClaw's broken built-in tool, a model-blocklist gate, and a ten-advisory security sweep. But the most uncomfortable part of the day wasn't the code. It was the README: the storefront for an open-source project, and ours was both underselling the work and quietly describing things the product doesn't do. Rewriting…