The agents are watching the book

TL;DR. Agents replaced what used to be a rigid-rule monitor plus quarterly portfolio reviews. The monitor watches the book daily and writes narratives. Our OpenClaw agent (Kai) posts the high-priority rows into Slack. Humans review and merge. Built on Anthropic Managed Agents, audited via GitHub PRs. Here's what we shipped, what broke, and what we'd ask the platform team for.

Here’s a thing nobody tells you about running a lending book: most of the work isn’t deciding who to lend to. It’s watching what happens after.

You write the check, the borrower goes off and runs their business, and now you have to keep tabs on a few hundred small companies whose health you can only see indirectly — through their bank data, their payment behavior, their occasional partner escalations. Distress almost never shows up as one dramatic event. It accumulates. Bank data goes stale because the borrower stopped logging in. A payment fails and then four more cluster around an extension request. Trailing revenue bends down for the third month in a row. Some other lender’s name shows up in the transaction feed. Each one of these is a faint signal. The hard part isn’t seeing any single one. It’s connecting them, every day, across the whole book, before anyone has to ask.

For most of the industry the way to do this looks like:

• A set of rigid rules running on a schedule. Utilization over X, days-past-due over Y. The output is a flat list that lands on someone’s desk every morning and needs human triage.

• A separate, much deeper review process that only kicks off when something triggers it — a credit-limit-increase request, a missed payment, a partner escalation.

• A learning loop that exists, technically, but moves at the speed of the quarterly portfolio review. Lessons from defaults get encoded into the next version of the rules eventually, when someone has the time.

This works. We did it this way for years. But it’s reactive by construction. The rigid rules catch what they were configured to catch and nothing else. The deep reviews are great but they only happen when someone trips a wire. And the learning loop closes at the speed of humans reading PDFs to each other — measured in quarters, not days.

The most operationally painful version of this is a class of signals nobody really monitors daily because the work is structurally too tedious to ever be anyone’s full-time job. Debt stacking is the canonical example. A borrower quietly takes on outside debt — usually an MCA, sometimes factoring, sometimes another short-term facility — on top of the financing they already have with you. Every lender knows this is a major distress signal. The usual treatment is to catch it at credit-limit-increase time, or after the fact in a postmortem, or via a UCC search run quarterly. To catch it daily you’d need a constantly-updated catalog of every competing lender — their names, their aliases, the dozen ways each one shows up in bank-transaction strings — joined against your cohort’s bank data on a schedule. Both pieces are tedious. Both drift constantly. Neither one is anyone’s full-time job at most credit shops. So nobody does it.

So the loud signal — this borrower just took on outside debt and is now showing strain — gets caught at credit-limit time, when somebody asks for more money. Not at the actual outside-debt event, weeks earlier. The data was always sitting in the warehouse. We just didn’t have the infrastructure to look at it daily.

This is exactly the shape of problem agents are good at.

Every weekday morning, before the credit team gets in:

1. An automated pipeline pulls the active borrower cohort.

2. Runs a battery of proprietary signals[^signals] against every borrower in it.

3. Writes a row per borrower to a watchlist table, with a severity bucket and a plain-English narrative explaining why it fired.

4. The high-severity rows show up in Slack, in the channel where the credit team already lives.

Steady-state output is small and actionable. A handful of P0s — call today. A few dozen P1s — review this week. A tail of P2s to monitor. Everyone else is marked clean. None of this is doing anything an analyst couldn’t do. It’s just doing it every single morning, automatically, with the reasoning attached, before anyone has to ask.

It’s an agent, not a rules engine. This part matters more than I expected going in. The thing writing the watchlist isn’t a hand-coded scoring function — it’s a Claude agent running on Anthropic Managed Agents that pulls the signal results, reasons about each borrower the way an analyst would, and writes the row with a narrative. “P0 because debt-service ratio is 47% and trailing revenue is down 18%” is a credit-analyst sentence. A rule engine literally cannot write that. The agent runs in Anthropic’s hosted environment, we never touch GPUs or model code, and the whole production system is about 1,200 lines of code.[^arch]

It’s PR-driven and auditable. This was the design choice that ended up mattering most, and not in a way I appreciated when we made it. Every change to what the monitor actually does — every new signal, every debt-provider entry, every threshold tweak, every prompt update — flows through our normal GitHub PR review process. The agent can propose changes (and does, daily, more on that in a sec). But a human reviews and merges. Nothing about the monitor’s behavior changes silently between runs. For a system writing to production data on a credit book, this is non-negotiable. I will die on this hill.

Debt stacking — the signal nobody monitors daily. Which brings us back to it. We solved debt stacking with a two-loop pattern. The first loop is the daily monitor, which joins the cohort’s bank transactions against a catalog of known debt providers and fires when a borrower took on new outside debt in the last 30 days. Severity scales with the type — MCA is a P0, factoring is a P1, credit cards don’t fire because they’re noise. The second loop is the catalog itself, and this is where the agent really earns its keep. There are tens of thousands of distinct payee names in the bank-data table. No analyst is going to classify those by hand. So a classifier agent runs every weekday morning, picks up whatever new names have shown up overnight, classifies them (this is an MCA, this is payroll, this is a SaaS subscription, this is a card payment), and auto-opens a PR with the proposed catalog updates.[^catalog] A human (me, today) reads the PR’s Markdown summary, approves the rows they agree with, merges. Days with zero new candidates produce zero PRs. The catalog grows on its own. The human stays in the loop on what gets promoted. Same review process as everything else.

Delivery via Kai, in Slack. A watchlist row that nobody reads doesn’t exist. That’s the last-mile problem and it’s the difference between “we built a thing” and “we built a thing that the team actually uses.” We solved it with another agent. Kai is our internal Slack bot, built on top of our home-grown OpenClaw runtime. Every morning Kai reads the day’s watchlist out of the warehouse, formats the P0 and P1 rows with the narratives the monitor agent wrote, and posts to a dedicated channel. Threads let the team discuss each borrower inline, tag the right account manager, close items out. The monitor writes; Kai reads and surfaces; humans act. Two agents, one workflow, no human in the middle just to ferry rows from one system to another. Agents everywhere.

What’s next. The monitor is just the first leaf of a bigger system. The three pieces still to build:

• PredictionChecker. Every day, looks at past watchlist rows whose 30/60/90-day horizons have matured and records whether each predicted borrower actually went late or defaulted. Also catches new late or defaulted events that weren’t on any prior watchlist — the misses. This is what closes the labeling loop. Until you have ground-truth labels you literally cannot talk about signal precision and recall.

• PostmortemAgent. For each missed default, pulls the borrower’s full data trail and writes a structured postmortem. What signals would have caught this in retrospect, what would need to change. This is where the model actually has something to add that a fixed-rule system can’t: synthesizing across a full borrower history to surface what a thoughtful human reviewer would have noticed in hindsight.

• InsightsLearner. Reads across postmortems, proposes new signal definitions or threshold tweaks. Opens a PR. Human reviews and merges. The system rewrites its own playbook.

Predictions → outcomes → postmortems → new signals → better predictions. Today we have the first arrow. Plan is to land one new arrow every few weeks until the loop closes.

This is the part that’s genuinely new compared to the old way of doing this work. Rigid rules don’t learn. Quarterly portfolio reviews learn slowly and lossily — the lessons live in whoever was in the room. A closed loop, running daily, that writes its own postmortems and proposes its own signal updates is just a different category of system. It’s not “rules plus humans plus a review cadence.” It’s something else.

A few things we picked up along the way that might generalize beyond our specific problem:

• The unlock from agents isn’t the model. It’s the workflow. We launched with three trivial signals. The infrastructure to deploy a new one is now one prompt edit plus one row in a catalog table. We’ve shipped dozens of signals in a few weeks. We never touched the model, never trained anything, never bought GPUs. The leverage is in how the work composes, not in some clever fine-tune.

• PR-as-notification beats dashboard-as-destination. Daily jobs that might produce work need to push, not pull. A PR opens with a summary in the body and the proposed diff. Email lands. You scan in 30 seconds. No dashboard to remember to open. No tab you keep forgetting to refresh.

• One agent’s output is another agent’s input. Monitor writes the watchlist. Kai reads it and posts to Slack. The classifier proposes updates that humans merge into what the monitor reads next time. None of them individually is doing very much. The leverage is in the composition.

[^signals]: I’m not going to list out the signal battery. It’s a meaningful chunk of our edge and I’d rather not telegraph the playbook. The shape is what you’d expect: a mix of utilization, repayment behavior, bank-data trend, exposure, and concentration, each with severity weights, each tuned over time against actual defaults in our own book. See Under the hood for how the battery is structured architecturally.

[^arch]: Architecture, design goals, and how the daily run actually executes are in Under the hood.

[^catalog]: The catalog-building mechanics — how the discovery pipeline aggregates per-name stats, how we cold-started it, how the daily steady-state runs — are in Under the hood.

Architecture, what broke, and what we’d ask the Anthropic team for.

Before any code got written we agreed on three design goals. Most of the technical choices fall out of them.

1. Scoped capabilities. The agent gets a narrow, well-defined tool surface — pull pre-computed signal results, write watchlist rows. No ad-hoc data access. No way to reach other systems. No internet. The trust boundary lives in the tooling, not in the agent’s good behavior. I don’t want to find out the hard way that the agent decided to be creative.

2. Auditable and PR-driven. Every change to what the monitor does — signals, thresholds, debt providers, prompt updates — flows through normal GitHub PR review. The agent proposes, humans review and merge. Always.

3. No raw data in the model’s context. All aggregation happens in the data layer. The agent reasons over compressed summaries — a row per borrower with the numbers that matter — not over millions of underlying transactions. Warehouse compute is cheap. Token budget is not.

The system has three actors. The warehouse is the source of truth — bank-data aggregates, issued products, status histories, loan-tape facts, repayment events all live there, along with a dedicated schema for the monitor’s own state. The agent (Claude Opus 4.7 on Managed Agents) runs in Anthropic’s hosted environment. And a small harness, about 250 lines of Python plus a couple of GitHub Actions workflows, deploys the agent, kicks off the daily session, polls to completion, and validates the output.

A daily run is unspectacular by design. A cron fires before the credit team is awake, the harness opens a session, the agent works through the cohort for twenty or thirty minutes — pulling signal results, scoring each borrower, writing the watchlist with narrative — and exits. By the time anyone starts their morning, the watchlist for the day is in the table.

We rolled this out in stages. Thin proof-of-concept first, to make sure the deploy → session → signal-pull → write → row-in-table loop actually worked end to end. Then the full proprietary battery. Then debt stacking.

The structural point that matters here: every signal in the battery is computable as a single batch query against tables that already exist. The agent pulls a handful of pre-computed summaries, each returning at most one row per cohort borrower with the relevant numeric context. The agent never sees raw transactions.

This is what makes the whole thing tractable economically. The agent’s working memory holds tens of KB of input across all signal results, not hundreds of MB of raw data. Adding signals adds warehouse compute (cheap) and a few hundred tokens (cheap). Not gigabytes of context.

The naive approach is “have the LLM look at every recurring debit name and classify it.” But the deduped transaction table has tens of thousands of distinct candidate names across the book. No single LLM session is going to fit that many through its context.

What worked was a two-pipeline split. A data-layer job for discovery. The agent for classification. A candidates table as the queue between them.

The discovery pipeline runs in the data layer and aggregates per-name stats. How many borrowers paid this entity. What the average payment cadence looks like (daily ≈ MCA, monthly ≈ card). The size distribution. The bank-data provider’s own debt/category tags. A few sample raw name strings for disambiguation. One row per distinct normalized name, with all the context an LLM would need. Names already matched by an active alias get filtered out, so the queue only contains unknowns.

For the cold start we dumped the queue to a CSV, wrote a hand-coded rule-based classifier for the obvious patterns, and let it bulk-classify the easy majority. The agent handled the ambiguous remainder interactively. End state was a seed catalog of about a hundred active debt providers covered by a few hundred alias patterns, with the long tail left pending for the daily catcher to handle as names accumulate weight.

For the steady state, the daily classifier reads the new pending candidates, classifies them, and auto-opens a PR. An operator reviews the Markdown summary, approves rows they agree with, merges. The PR is the notification mechanism — email arrives when there’s new work, days with no candidates produce no PR, no noise. Same review process as everything else.

Three painful failures along the way. All the same shape.

Failure 1. First production run tried to write 305 rows in a single tool call. Got back Missing required argument: rows. Assumed our internal tool had a bug. Spent an evening enumerating session events via the SDK, only to find that the agent’s tool_use input had three of the four required keys but no rows — even though the preceding agent.message clearly said “writing 305 rows now.”

Failure 2. Tightened chunk size to 50. Worked for the initial small signal set (305 rows × ~250B narrative). Failed again once we expanded the signal battery. Same row count, but per-row payload tripled because the narratives now concatenated multiple signal fragments.

Failure 3. Tightened chunk size to 20. Works now. But only because we hand-tuned the constant against three failed production runs.

Same root cause every time: the model’s tool_use output hit max_tokens mid-stream. The JSON couldn’t close cleanly. The agent’s next turn saw a generic schema validation error (”missing required argument: rows”) rather than a truncation signal. The underlying stop_reason: "max_tokens" is right there in the Messages API response — the docs literally describe this exact case as “incomplete tool_use” — but it doesn’t get propagated through the agent harness into the agent’s own context in a way the agent can act on. So it retried with the same payload. Then it timed out.

This is the single most painful failure mode we’ve hit. Three independent occurrences, none of which the agent could have recovered from on its own, all of which required us to enumerate raw session events to diagnose. Each cost us an evening.

We’re production-using Managed Agents and broadly love the platform. Two weeks of evenings to get from CLI script to production daily cron is real. The platform has also moved fast since we started building — native multi-agent orchestration and the session dashboard with full tool-call drill-down in the Console both landed while we were building this. So this list is what’s still open from where I sit.

The Messages API already exposes the signal. When generation stops with stop_reason: "max_tokens" and the last content block is an incomplete tool_use, that’s the documented condition we hit. The docs even prescribe the handling pattern. The gap is at the harness layer above it. When this happens inside a Managed Agents session, what the agent sees on its next turn is a Pydantic-style “missing required argument: rows” error — the same error it would see if it had genuinely forgotten a parameter. So it retries identically. Which fails identically.

What would have saved us those three evenings: when the harness detects stop_reason: "max_tokens" on a tool_use block, surface that as a typed error back into the agent’s context — “your previous tool_use was truncated at max_tokens; the call did not run; reduce payload and retry” — rather than as a generic schema validation failure. The signal already exists at the model layer. The ask is for the harness to propagate it without ambiguity.

To preempt the obvious pushback: this isn’t the session-level usage field (we have that, it’s cumulative input/output tokens for the operator). It isn’t Task Budgets either (which caps overall agent effort, also operator-facing). And it isn’t solved by fine-grained tool streaming — that reduces buffering latency and lets you observe partial JSON deltas, but the model still hits max_tokens and the JSON is still incomplete. You just see the truncation point earlier in the deltas instead of inferring it from a missing field after the fact.

What we actually want is different: a number the model itself can see mid-reasoning. Something like output_budget_remaining_tokens exposed in scope, so the agent can compute its own safe chunk size from the actual payload shape before emitting the tool call. Kills a class of hand-tuned constants in agent prompts. Complements (1) — (1) recovers gracefully when truncation happens, (2) prevents it from happening in the first place.

Today we declare a JSON schema for the agent’s final message in the harness, parse the agent’s text output, and validate. About half of run_session.py is plumbing for this. An agent-config-level output_schema that Anthropic enforces (and that surfaces validation errors back into the agent’s context on retry) would shrink the harness considerably and make outputs more reliable.

Three takeaways for anyone building something similar:

1. Don’t put raw data in the agent’s context. Aggregate in your data layer, let the agent reason on summaries. Our agent never sees a raw transaction. It sees MAX(DATE) per borrower. Warehouse compute is cheap. Token budget is not.

2. Per-row payload size is the binding constraint, not row count. A 305-row batch that worked at 250B/row failed when the row payload grew to 1000B/row. If you can’t predict it ahead of time, bake chunk-sizing into the prompt rather than relying on a tuned constant — and when the model gives you output_budget_remaining_tokens someday, switch to that.

3. A warehouse review queue beats an artifact pipeline for human-reviewed agent output. We started with the discovery agent emitting JSON artifacts uploaded to GitHub Actions. Moved to a candidates table in the warehouse. The table lets you query, sort, filter, slice the agent’s proposals using existing muscle memory. JSON artifacts mean tar-extract-and-grep.

If you’re doing similar work and run into the same failure modes, or have opinions on what to build next — please reach out. Always happy to compare notes.