Benchmark · 2 September 2026
The same job, run twice.
Everything about an effect gate is a counterfactual — the charge that did not happen. You cannot see it, so the only honest way to make the claim is to run the identical job with the gate and without it, and count what a third party actually received.
The setup
A vendor that always does the work, and sometimes fails to say so.
A local server stands in for a payments API. It records every refund it actually performs, and that record is the ground truth — not what the agent believes, and not what Ratchet reports.
The failure simulated is the one people under-model: the request arrives, the vendor executes it, and the response is lost on the way back. The caller sees a timeout and cannot tell it apart from a request that never landed. Retrying is correct behaviour. Retrying is also what charges the customer twice.
- 40
- refunds of $240.00, one per customer
- 25%
- of responses lost in flight
- 3
- retries, which is what a well-built agent does
- 1
- seeded failure sequence, identical in both runs
Request #3 fails identically in both runs. Without that, the comparison measures luck.
Forty jobs, both ways.
scroll to runWhat it cost
Thirteen refunds that should not exist.
Eleven customers were refunded more than once. One was refunded four times. Every block is $240 of somebody's money, moved again for work the vendor had already done.
The result
What the vendor received
| Measure | Without the gate | With the gate |
|---|---|---|
| Refunds the vendor performed | 53 | 40 |
| Distinct customers refunded | 40 | 40 |
| Duplicate refunds | 13 | 0 |
| Money moved | $12,720.00 | $9,600.00 |
| Money that should have moved | $9,600.00 | $9,600.00 |
| Overpaid | $3,120.00 | $0.00 |
Identical across consecutive runs, because the failure sequence is seeded. The correctness result is not a sample — it is deterministic.
Getting there
The p95, before and after.
Both bars run in real time. The slow one is a request waiting on database round trips that could not overlap; the fast one is the same request after five of them were removed.
What was actually slow
The gate was never CPU-bound. It was waiting on the database, in series.
Signing, canonicalisation and hashing total about 0.2 ms. Everything else was sequential awaited round trips that cannot overlap. Counting them beat guessing at them.
16 queries per replay → 9
| What was happening | Saved |
|---|---|
| Authentication ran twice per request — the rate limiter needed the caller's plan, then each route guard authenticated the identical string again | 1 query + HMAC |
| A duplicate asked about the same row three times: a pre-check, an INSERT guaranteed to conflict, and a re-SELECT to learn what the row said | 2 queries |
Opening a transaction took two round trips before doing anything — BEGIN and SET LOCAL now travel together | 1 round trip |
| The surge baseline was read for most new effects to compute a number nothing read | 1 query |
Reserving spend issued nine sequential statements — now one INSERT … unnest … ON CONFLICT DO UPDATE … RETURNING | 8 statements |
What it costs
Asking permission is indistinguishable from an empty HTTP request.
Measured warm, inside the datacentre, with performance.now(). The comparison
that matters is the last row: a request that does nothing at all.
The gate does less work than the network it arrives over.
An unexpected result
The gated job finished faster than the ungated one.
By 64–269 ms per job, across three runs. That is not the gate being free — it is a prevented duplicate being worth more than the gate costs. Every duplicate refused is a vendor call that never happens, and in this workload a failing vendor call costs a four-second timeout against roughly 60 ms to ask permission.
This depends on the vendor being slow. Against a fast vendor the gate would add net time. It is stated because an earlier run showed the same effect, it was nearly reported, and the next run reversed the sign. It was variance then. It became a finding only when it held across three runs with a mechanism that explains it.
What this does not show
The parts a benchmark page usually leaves out.
A 25% loss rate is severe
It was chosen to produce a legible result in 40 jobs, not because it is typical. At 1% the duplicate count falls roughly proportionally. The mechanism is identical; only the frequency changes.
This vendor has no idempotency of its own
Against a vendor that supports idempotency keys and is used correctly, the duplicate would be refused at the far end instead. Ratchet issues that key too — but the point of the gate is the vendors that do not offer one, which is most of them.
Some of the p95 gain is a warm machine
The first version of this benchmark reported the gate costing 25 ms. It was measuring a suspended machine waking up. That confound is named here rather than claimed as credit.
Exactly-once is not claimed, here or anywhere
It is not achievable. Ratchet guarantees at-most-once initiation,
enforced by a database unique index rather than by application logic — and when a
lease expires unreported, the outcome stays indeterminate rather than
being guessed.
Reproduce it
The harness is in the repository.
Same seed, same sequence, same result. The vendor's own execution log is the ground truth, and it is printed at the end of every run.
# against a local stack, or set RATCHET_URL for production
npm run bench:ab