Should retries and timeouts live in your application or your service mesh?

Photo by Jordan Harrison on Unsplash.

Should retries and timeouts live in your application or your service mesh?

This debate comes up constantly.

Should resiliency live in the platform components, or should the application own it?

Like most things in distributed systems, the answer is: It depends.

Service meshes, API gateways, and load balancers are great at handling generic resiliency concerns.

Things like:

• Connection timeouts

• Automatic retries

• Circuit breakers

• Request-level failover

The advantage is obvious. You remove complexity from the application and apply resiliency consistently across services.

For many scenarios, this is the right answer.

The challenge is that platform services only understand traffic. They don’t understand why the request is made.

They don’t know whether a request is reading customer profile data, reserving inventory, transferring money, or uploading cat videos.

To the service mesh, they’re all just requests. Some requests might even have custom timeouts and retries configured. Most of the time, that’s fine.

Sometimes, it’s not.

Some resiliency decisions require application context.

Consider a financial transaction. Should a timeout trigger a retry? Maybe.

Does the request have an idempotency key? If yes, blindly retrying might be safe. If not, things become more complicated.

Did the request reach the downstream system? Was it partially processed?

Do you need a compensating transaction before retrying?

At this point, the retry is no longer a networking decision. It’s a business decision. And business decisions belong in the application.

The best architectures usually use both approaches.

Use platform resiliency whenever the decision can be made without application context. But when correctness depends on understanding the request itself, move that logic into the application.

There should be a strong preference toward offloading complexity when possible. Just don’t offload decisions that require business context.

I often see teams over-index on platform-level resiliency. And while reducing application complexity is valuable, it isn’t free.

Infrastructure understands traffic and request characteristics. Applications understand business intent.

When resiliency decisions depend on business intent, they belong in the application.

Originally posted on #Bengineering.

If this hit home, share the original with someone who needs it: Should retries and timeouts live in your application or your service mesh?

• Glue Services: Part Two — Data Synchronization
  A useful companion on keeping old and new systems aligned during gradual platform change.

• When modernizing legacy systems, don’t be afraid to build glue services
  Edge translation and isolation patterns pair well with migration-focused architecture decisions.

• YOLO Is a Terrible Strategy for Validating Production Changes
  Operational changes get safer when validation and rollout discipline are part of the design.

• Coding agents can’t see your architecture diagrams—fix that
  Systems thinking improves when design context stays readable, versioned, and close to the code.

• In-Flight Request Tracking: Lessons from Card Payments and HTTP/2
  A deeper look at uncertainty, timeouts, and correctness when distributed requests do not complete cleanly.

• Are Atomic Operations Faster and Better Than a Mutex? It Depends
  A practical reminder that simpler-looking performance choices still need correctness tradeoff analysis.