Idempotent payment capture

Author: Tomas Berg · Date: 2026-09-08

Overview

Halden Labs' billing service sometimes charges a customer twice when a client retries a capture request after a network timeout. This spec adds idempotency keys to POST /v2/captures so a retried request returns the original result instead of creating a second charge.

Goals

• A retried capture with the same Idempotency-Key never creates a second charge.
• Duplicate captures drop from about 40 a month to zero within one release.
• p95 latency of the capture endpoint rises by no more than 10 ms.

Non-goals

• Idempotency for refunds and payouts (follow-up spec).
• Changing the payment processor integration.

Approach

Store each key with a request hash and the response in a new idempotency_keys table, written in the same transaction as the capture. A unique constraint on the key makes concurrent retries safe without a distributed lock.

Detailed design

New middleware idempotency.ts in billing/http reads the header, hashes the body, and either replays a stored response or reserves the key. CaptureService.capture() takes the reservation and stores the response before commit. Keys expire after 24 hours via the existing cleanup job.

sequenceDiagram
    participant C as Checkout client
    participant A as Billing API
    participant S as CaptureService
    participant D as Postgres
    C->>A: POST /v2/captures (Idempotency-Key)
    A->>D: Insert key as reserved
    alt key already completed
        D-->>A: Stored response
        A-->>C: 200 replayed
    else new key
        A->>S: capture()
        S->>D: Insert charge and response in one transaction
        S-->>A: Charge
        A-->>C: 201 Created
    end

A key reused with a different body returns 422 idempotency_mismatch. A key still reserved by an in-flight request returns 409 so the client retries later.

Sequencing

1. Migration for idempotency_keys (no reads yet).
2. Middleware behind flag capture_idempotency, off.
3. Checkout clients start sending the header.
4. Flag on at 10%, then 100%.

Milestones

Testing strategy

Unit tests cover replay, mismatch and in-flight cases. An integration test fires 20 concurrent identical captures and asserts exactly one charge. It does not prove behavior under a Postgres failover; that is covered by the staging chaos run.

Risks

Open questions

• Keys scoped per merchant, not global (decided in review).
• Expiry window: 24 hours, matching the processor's retry window.

Review

Review by: 2026-09-12