Keep Transactions Fast, Not Fragile

Subscribe now

Why external API calls and database locks should never share the same critical section in 2026

A lot of production bugs start from a choice that feels perfectly reasonable when you first write it. You need to perform a few calls to Stripe, collect the resulting objects, and store everything in your database. Naturally, you want the operation to behave like one unit: if something goes wrong, you do not want half the records hanging around. So you wrap the whole workflow in a database transaction and move on.

That instinct is understandable. It also creates a nasty failure pattern. ⚠️ Putting third-party network requests inside a database transaction often looks tidy in code review, yet it can punish you badly under real concurrency. The database ends up holding locks while your process waits on an outside service and the public internet. That mismatch is where trouble starts.

Let’s walk through the tempting version first, then the safer structure, and finally the edge cases that still matter even after you fix the lock problem.

🧠 The Version That Feels Correct

Imagine you are scheduling a payment. The flow needs three Stripe interactions:

1. Create a customer 2. Attach a payment method 3. Create a charge

After each response, you want to persist the corresponding database row.

def call
  ActiveRecord::Base.transaction do
    customer = Stripe::Customer.create(email: user.email)
    db_customer = StripeCustomer.create!(stripe_id: customer.id, user: user)
    payment_method = Stripe::PaymentMethod.attach(
      payment_method_id,
      customer: customer.id
    )
    db_payment_method = StripePaymentMethod.create!(
      stripe_id: payment_method.id,
      stripe_customer: db_customer
    )
    charge = Stripe::PaymentIntent.create(
      amount: amount_in_cents,
      currency: 'usd',
      customer: customer.id,
      payment_method: payment_method.id
    )
    StripeCharge.create!(stripe_id: charge.id, stripe_payment_method: db_payment_method)
  end
end

At a glance, this seems clean. If Stripe::PaymentIntent.create raises an exception, the transaction rolls back and your local database avoids partial state. ✅ That part is true.

The real problem is everything happening while the transaction stays open.

⏱️ The Lock-Time Mismatch

A database transaction is not free. Depending on the isolation level and the specific statements involved, it can keep row locks, and in some cases broader locking effects, for the entire transaction lifetime.

Inside a healthy system where services sit close together, a basic SELECT, INSERT, or UPDATE should usually complete in well under 10ms. Even multiple writes bundled into one transaction are often still below that threshold.

Now compare that to a typical Stripe request. Those calls frequently land in the 500–700ms range, and sometimes longer. That delay is outside your direct control. You can optimize your own infrastructure all day and still be stuck waiting on a remote vendor plus network latency. 🚦

So if your transaction contains three Stripe calls, you are no longer talking about a quick local critical section. You are holding database locks for something like 1.5 to 2+ seconds. Under concurrency, that is enough to create contention, stalled queries, backed-up workers, and eventually widespread slowdown.

That is the core issue: you are forcing the database to wait on the network.

🏭 What This Turns Into in Production

Here is how this sort of bug shows up for real. A background job runs every morning, updates a large amount of cached data, makes several external API requests, and writes many rows. Some version of that workflow ends up placing the network calls inside transactions.

Then the morning traffic arrives.

Connections stay occupied longer than expected. Writes compete with other writes. Queries start queueing behind locks. What began as one “atomic” workflow becomes a source of cascading database pressure at the worst possible moment. 📌

Once the issue is identified, the remedy is usually simple: stop doing network I/O inside the transaction.

🛠️ The Better Structure: API First, Transaction Second

A safer pattern is to split the command into two phases:

• First, make every external API call

• Second, open a transaction only for database writes

Store the API results on the command object, then persist them in a short transaction.

class SchedulePaymentCommand
  def call
    fetch_stripe_objects # pre-transaction step: all API calls happen here
    persist_to_database # transaction step: DB writes only, no network I/O
  end
  private
  def fetch_stripe_objects
    @stripe_customer = Stripe::Customer.create(email: user.email)
    @stripe_payment_method = Stripe::PaymentMethod.attach(
      payment_method_id,
      customer: @stripe_customer.id
    )
    @stripe_charge = Stripe::PaymentIntent.create(
      amount: amount_in_cents,
      currency: 'usd',
      customer: @stripe_customer.id,
      payment_method: @stripe_payment_method.id
    )
  end
  def persist_to_database
    ActiveRecord::Base.transaction do
      db_customer = StripeCustomer.create!(
        stripe_id: @stripe_customer.id,
        user: user
      )
      db_payment_method = StripePaymentMethod.create!(
        stripe_id: @stripe_payment_method.id,
        stripe_customer: db_customer
      )
      StripeCharge.create!(
        stripe_id: @stripe_charge.id,
        stripe_payment_method: db_payment_method
      )
    end
  end
end

This version uses the transaction for what it is good at: coordinating quick, local writes. Lock duration drops from roughly 1.5 seconds to under 10ms. 🚀

That difference is enormous once the system has multiple workers or overlapping requests.

❓ “But Doesn’t This Lose Atomicity?”

This is the obvious pushback, and it is a fair one.

If the API calls succeed and the database write later fails, then yes, the remote side may already contain created objects. But here is the important point: the original “all inside one transaction” approach never actually solved that problem either.

A database rollback only undoes your local writes. It does not reverse calls that already succeeded on Stripe’s side. If you created a customer, attached a payment method, and created a payment intent before a later DB failure, those remote side effects still happened. The rollback merely hides the inconsistency in your own storage. ⚠️

So the wrapped transaction gives you less safety than it appears to.

What you actually want is explicit cleanup logic.

🧹 Replace False Safety With Real Cleanup

A more honest design is to define a failure path that knows how to undo the remote work the command performed.

class SchedulePaymentCommand
  def call
    fetch_stripe_objects
    persist_to_database
  rescue => e
    handle_error(e)
    raise
  end
  private
  def fetch_stripe_objects
    @stripe_customer = Stripe::Customer.create(email: user.email)
    @stripe_payment_method = Stripe::PaymentMethod.attach(
      payment_method_id,
      customer: @stripe_customer.id
    )
    @stripe_charge = Stripe::PaymentIntent.create(
      amount: amount_in_cents,
      currency: 'usd',
      customer: @stripe_customer.id,
      payment_method: @stripe_payment_method.id
    )
  end
  def persist_to_database
    ActiveRecord::Base.transaction do
      db_customer = StripeCustomer.create!(
        stripe_id: @stripe_customer.id,
        user: user
      )
      db_payment_method = StripePaymentMethod.create!(
        stripe_id: @stripe_payment_method.id,
        stripe_customer: db_customer
      )
      StripeCharge.create!(
        stripe_id: @stripe_charge.id,
        stripe_payment_method: db_payment_method
      )
    end
  end
  def handle_error(error)
    # Cancel whatever was created on Stripe's end, if anything
    Stripe::PaymentIntent.cancel(@stripe_charge.id) if @stripe_charge
    Stripe::PaymentMethod.detach(@stripe_payment_method.id) if @stripe_payment_method
    Stripe::Customer.delete(@stripe_customer.id) if @stripe_customer
    Rails.logger.error("SchedulePaymentCommand failed: #{error.message}")
  end
end

This gives the command a precise responsibility: if the operation fails after creating remote artifacts, handle_error knows what to reverse. ✅

An important detail here is architectural. handle_error is meant to be specialized by subclasses. It is not generic rescue glue. It is the intentional reversal mechanism for a command that knows what it created.

That is a much better contract than pretending a DB rollback can somehow rewind another system.

🧨 The Harder Edge Case: Process Death Mid-Flight

There is another failure mode worth discussing because it exists whether you use the old pattern or the improved one.

Suppose a container is shutting down during a deploy. The orchestrator sends SIGTERM, your app starts draining requests, and ideally everything finishes cleanly. In reality, the shutdown window may be too short, the drain behavior may be flawed, or the process may get killed before in-flight work completes. 💥

Now imagine the timing is awful:

• the request to Stripe has already been sent

• Stripe completes it successfully

• your process dies before it receives and stores the response

From Stripe’s perspective, the object exists. From your database’s perspective, nothing happened. That leaves you with a “ghost” side effect: a real external mutation with no matching local record.

If the job retries without remembering the earlier attempt, it will send the same calls again. Without idempotency keys, Stripe treats them as separate requests. In the case of Stripe::PaymentIntent.create, you can end up with two charges for one order. That is the kind of bug people remember. ⚠️

🔐 Idempotency Keys Are Mandatory

The standard defense is to use deterministic idempotency keys per logical operation. A common strategy is to derive them from something stable such as the order ID plus the action type.

def fetch_stripe_objects
  idempotency_key = "schedule_payment_#{order.id}"
  @stripe_customer = Stripe::Customer.create(
    { email: user.email },
    { idempotency_key: "#{idempotency_key}_customer" }
  )
  @stripe_payment_method = Stripe::PaymentMethod.attach(
    payment_method_id,
    { customer: @stripe_customer.id },
    { idempotency_key: "#{idempotency_key}_payment_method" }
  )
  @stripe_charge = Stripe::PaymentIntent.create(
    {
      amount: amount_in_cents,
      currency: 'usd',
      customer: @stripe_customer.id,
      payment_method: @stripe_payment_method.id
    },
    { idempotency_key: "#{idempotency_key}_charge" }
  )
end

With these keys, a retry after an interrupted execution becomes safe. Stripe sees the repeated key within its 24-hour idempotency window and returns the original result instead of creating another object. 🛡️

That solves duplicate remote objects.

It does not completely solve durability.

🧩 What Idempotency Still Does Not Cover

Even with idempotency, there is still a gap between “remote side effect completed” and “result was stored locally.” If the process dies in that window, you may lose the original response from the first attempt. The retry can fetch the same result again, but the system still experienced a silent interruption point.

A stronger answer likely requires making the pre-transaction phase itself more durable, such as recording intent in the database before contacting the external system, then reconciling partial progress during retries. That is a bigger pattern and deserves dedicated treatment. 📌

For the problem discussed here, though, the rule stays the same.

✅ The Practical Rule

A database transaction should include only database work.

If a call crosses the network boundary, keep it outside the transaction. Do the external requests first. Validate the responses. Then persist the local rows inside a short, focused transaction. If failure happens after remote state was created, handle that explicitly with cleanup logic designed for the command.

Transactions are excellent for fast, reliable, local coordination. They are not meant to stretch across slow and failure-prone network operations. When you try to make them cover both, you usually get the worst of both worlds: more fragility and less throughput. 🚀

🔍 TL;DR Summary

• ✅ Keep third-party API calls out of database transactions to avoid long lock times and contention.

• ⏱️ Local DB work often finishes in under 10ms, while Stripe calls commonly take 500–700ms each.

• ⚠️ Wrapping network calls in a transaction does not provide true atomicity because rollback cannot undo remote side effects.

• 🧹 Use explicit cleanup methods to reverse external state when later steps fail.

• 🔐 Add deterministic idempotency keys to prevent duplicate remote objects during retries or container shutdown races.

• 📌 For stronger recovery, consider durable intent recording before external calls.

Subscribe now