We Ran the Exit Test on Ourselves

Last week I wrote down where we draw the line between what a client owns and what we keep, and Skip added a test for it: ask a vendor what you are holding the day after you leave, and watch how fast the answer comes. I closed by saying we would run that test on our own systems and show you the parts where our answers are less clean than we would like.

I have spent fifteen years watching teams pass audits they should have failed, and the reason is almost always the same.

The test measures whether an artifact exists. It does not measure whether the artifact is true.

So when I ran our own exit test, the interesting part was not the score. It was the set of things the test never asks about.

Three findings. One of them is uncomfortable enough that I would rather publish it than have somebody find it.

Run the question on us and most of it resolves quickly, which is what a good architecture is supposed to do.

The context layer lives in our own store, on infrastructure we control, and it is readable without us. Written intent and decision records are text in a repository and issues in a tracker, both of which outlive any particular vendor relationship including ours. The agents run against our own credentials. Nothing sits in the middle that we could switch off, because we deliberately built nothing that sits in the middle.

That answer came fast, which by Skip’s own standard is the tell that we had to decide it in order to build anything. Fine. Here is what the fast answer conceals.

Every artifact in that list is portable. Not every artifact in that list is current.

The exit test asks what you keep, and a document you keep is worth exactly as much as its agreement with the running system. We have decision records explaining choices made against a version of the architecture that has since gained a whole surface. They are accurate about why we decided what we decided. They are increasingly incomplete about what is actually deployed. Nothing in the exit test catches that, because the file is right there where it is supposed to be.

The test measures possession. It does not measure freshness. If you run it on a vendor, ask the second question too: when was the last time this record was written to, and by what.

Here is the number I did not want to publish.

Our own operating system has a feedback project like any other product we run, and anyone on the team can file into it from inside the app. As of today it holds nine open items. Six are bugs, three are feature requests. The oldest open bug was filed on the twenty-fourth of July, which makes it a month old. The newest arrived yesterday.

In that same window we closed three. One was fixed eleven minutes after it was reported. One was fixed the next day, after a policy confirmation from the person who actually knows the labor code in question. The third took twelve days.

Look at what separates the two groups and it is not severity, it is friction.

The three that closed were all blocking a specific person from finishing a specific task that morning. A date column that did not show the year, so nobody could tell which year a record belonged to. A statutory sick-leave entitlement carrying the wrong number of hours for one country, which is exactly the kind of defect a payroll conversation surfaces immediately. A record that had to be recategorized before a month could be closed.

The six still open are the ones where nobody is standing still. A question about whether a prior year of data survived a migration. An accrual policy that appears to be resolving to one country’s calculation in a place where the logic should probably live somewhere else. A pipeline record that will not convert to the next state. A menu badge that says one request is waiting while the screen behind it says zero.

None of those stop anyone from working today, so none of them get escalated. And there is nobody to escalate to. The mechanism that makes a vendor fix your bug is that you might leave. When you own the layer, that mechanism is gone, and what replaces it is whatever discipline you actually have rather than the discipline you believe you have. We have written about the gate that decides when something is done. We do not have an equivalent forcing function for deciding when something stays broken.

That is the real cost of owning your layer, and it is not the one anybody budgets for.

It is not the build. It is that you have just acquired a client who never threatens to churn.

Skip closed a piece three weeks ago by saying the pattern that already runs our time-off flow would extend outward to the money, the pipeline, and the outward-facing systems, and called it what we are building next. He has asked me to tell you exactly where that is, so I am going to, including where the word we used was wrong.

What is shipped. An MCP server over our internal access model. It exposes read tools for resolving who has access to what, and write tools for granting and revoking it. The thing worth saying about it is not the tool list. It is that the server is a thin process over the service that already existed, so authorization and audit logging happen inside that shared service and not in the MCP layer. The admin check that guards the screen is the same admin check that guards the tool, because it is the same function. We wrote parity tests that run identical assertions against both surfaces, so the two cannot drift into different answers about who is allowed to do what.

What is designed and not built. The client-facing version. The tools were never the hard part there. Authorization was. We evaluated a third-party identity provider for it and decided against, which means we are running our own authorization server, federating identity up to the two enterprise providers our clients actually use, and scoping access at the company level by verified email domain plus a tenant claim pinned per client.

That is a decision, not a deployment, and I am not going to describe it in the present tense until it is one.

What does not exist is a unified MCP service. We used that phrasing and it was the wrong word, so I will correct it here rather than let it compound. There is no single endpoint that fronts the money, the pipeline, the delivery data and the outward-facing systems together, and I no longer think there should be. A monolithic layer that spans all of those would have to hold its own model of what each of them means, and the moment it does, it is a second copy of business logic drifting away from the systems it fronts. That is the specific failure the architecture exists to prevent. Building it into the newest layer because the newest layer is exciting would be a bad trade.

What is actually unifying is the pattern and the auth model, not the address. Each surface stays thin over its own system, with authorization resolved once, in the service that owns the data, and the same identity model across all of them.

That is less impressive to announce and considerably more likely to still be true in a year.

The staleness problem has no owner. We know our decision records lag the running system. We do not have a mechanism that closes the gap, and adding one is easy to write down and historically easy to abandon, because it fires on a day when nothing is broken.

Portability is not operability. Everything we would hand you is readable without us. Whether a new team could actually operate it in week one is a different claim and we have not tested it, because testing it properly means handing the system to someone who was not in any of the conversations and watching what breaks. I would like to run that. It is expensive and nobody is asking us to.

The asymmetry I named last week has a second edge. The reusable components improve because of client work. Our own instance is the one thing in the portfolio that no client is waiting on, which makes it the first thing to go stale and the last thing to get attention. We have been publishing for five weeks about a system we run our own business on. It is a real system, it does real work, and it also has a month-old bug in it that nobody has picked up. Both of those are true and I would rather you have both.

So the exit test still holds. Ask what you walk away with, and take a slow answer as the finding. Just do not stop there, because possession was never the whole question. Ask when the record was last written to, and ask who is on the hook for the system once nobody is billing for it. On our own stack the answer to that second one is us, and the honest version is that we are not always fast about it.

Next up: Skip has now told you twice to run the math on your own business and handed you no instrument. Next week he fixes that. The actual diagnostic, in order, with what each answer means.

If this resonated, subscribe. We’re writing about what’s actually changing in software delivery, no hype, no hand-waving, just what we’re seeing on real projects.