How preflight avoids double-billing under concurrent load

The check that reads

The obvious preflight check reads the job, adds the estimate, and compares:

# Read, then decide. Do not use this under concurrency.
def preflight(task_ref, estimated_usd):
    job = db.one("SELECT ceiling_usd, spent_usd FROM jobs WHERE task_ref = %s",
                 task_ref)
    if job.spent_usd + estimated_usd > job.ceiling_usd:
        return {"approved": False}
    return {"approved": True}

With one worker it works. Now give it five. The job has a $5.00 ceiling and has spent $4.90, and each call estimates $0.10:

worker 1: reads spent = $4.90. 4.90 + 0.10 <= 5.00. approved.
worker 2: reads spent = $4.90. approved.
worker 3: reads spent = $4.90. approved.
worker 4: reads spent = $4.90. approved.
worker 5: reads spent = $4.90. approved.

five calls go out. spent = $5.40 on a $5.00 ceiling.

Every read happened before any write, so every worker saw $4.90. This is a time-of-check to time-of-use race: the state changed between the check and the spend, and nothing in the check could see it coming. A lock in your own process doesn't help once the workers are separate processes, or separate machines.

The check that writes

AgentBill's preflight doesn't read the balance and then decide. It tries to reserve the estimate, and the condition is part of the write:

-- inside POST /preflight, in one transaction (simplified)
UPDATE task_budgets
SET reserved_units = reserved_units + :estimate
WHERE account_id = :account_id
  AND task_ref   = :task_ref
  AND used_units + reserved_units + :estimate <= ceiling_units
RETURNING ceiling_units, used_units, reserved_units

If the estimate fits, a row comes back, and the reservation is already in reserved_units where the next worker's UPDATE will see it. If zero rows come back, it didn't fit. The answer is approved: false with the reason task_ceiling_exceeded, nothing was reserved, and your code decides what happens next: wait, fall back to a cheaper model, hand the job to a person, or let it end there.

Postgres takes a row lock for the UPDATE, so the five workers queue on the job's row for the length of one statement each. There's no read-then-write gap left to fall into. On a job in dollars the numbers are micro-dollars, so $5.00 is 5,000,000 and a $0.10 estimate is 100,000. Same five workers:

worker 1: 4,900,000 + 0         + 100,000 <= 5,000,000. reserved. approved.
worker 2: 4,900,000 + 100,000   + 100,000 >  5,000,000. zero rows. approved: false.
worker 3: zero rows. approved: false.
worker 4: zero rows. approved: false.
worker 5: zero rows. approved: false.

one call goes out. the other four hear approved: false before sending anything.

What the job can still go over by is the gap between one call's estimate and what that call really cost. That's the next section.

Settling to what the call cost

When the provider answers, record reports the tokens it used. On a job in dollars the server prices those tokens at the provider's public list price and settles by that amount: the cost goes into used_units, the whole reservation comes out of reserved_units, and whatever the call didn't need is free for the next worker right away. A $0.10 reservation for a gpt-4o call of 10,000 input and 2,000 output tokens settles at $0.045.

So the estimate matters, and you don't have to guess it. Leave estimated_usd out and the server reserves the job's recent median call, or $0.10 before the job has a priced call. Once the job is at 80% of its ceiling, counting holds still in flight, it reserves the job's largest recent call instead: loose while there is room, strict where overshoot can actually happen. The answer names whose estimate it was in estimate_source: caller, job_median, job_max or default. A call that costs more than its reservation is still charged in full, because the spend happened, and spend that lands past the ceiling is recorded and flagged task_exceeded rather than dropped. The ceiling holds to within one call's miss, not to the cent. Every dollar figure here is an estimate at list price, not your provider's invoice.

A call that never reports back

When the provider call throws, wrap() releases the reservation with a zero-unit record that names it, so the money is back before the exception reaches your code. The harder case is a worker that goes away without a word: the process exits, the machine restarts, the network drops. Nothing will ever settle that reservation.

So every reservation expires. Each approved preflight returns reservation_expires_at, 60 minutes out on the hosted service, and a sweeper reclaims the ones that pass it. That needed one change to the shape of the data. reserved_units is a counter, and a counter can't be swept, because it doesn't know how much of itself is stale. So a reservation is also a row, and the counter is the sum of the open rows:

-- The invariant every path keeps
task_budgets.reserved_units = SUM(units) of the job's open reservation rows

Which makes the sweep boring, and boring is the goal here:

-- The sweeper's claim, trimmed. Expired rows and their
-- units go back in ONE transaction. SKIP LOCKED because
-- production runs two machines, and both sweep.
UPDATE reservations SET released_at = now()
WHERE id IN (
  SELECT id FROM reservations
  WHERE released_at IS NULL AND expires_at < now()
  ORDER BY expires_at LIMIT 500
  FOR UPDATE SKIP LOCKED
)
RETURNING id, account_id, task_ref, units

The late settle

Now two different things can close the same reservation: the sweeper, and a record that arrives late. A queued retry, a slow worker, a caller that held on to the id for an hour. If the record takes its reservation off the counter no matter what, those units come off twice, once by the sweeper and once by the settle. The counter then sits below what's really in flight, and preflight starts approving calls against money another call is still holding. A double release is a double spend.

So a record doesn't take off what it thinks was reserved. It closes the reservation it names, and takes off what that row was still holding, which is nothing if the sweeper got there already:

-- record() names the reservation its preflight returned
UPDATE reservations SET released_at = now()
WHERE id = :reservation_id AND released_at IS NULL
RETURNING units                  -- :returned, or no row (0) if the sweeper took it

-- then, in the same transaction
UPDATE task_budgets
SET used_units     = used_units + :cost,
    reserved_units = GREATEST(0, reserved_units - :returned)

The cost always goes in, because the call really ran. The reservation comes out once, whichever of the two got there before the other.

The retry that reserved twice

One more hole, and it was in the mechanism that exists to prevent waste. /events has enforced (account_id, idempotency_key) as unique from the start. /preflight had nothing, so a client that retried a timed-out preflight reserved a second time, and an eager retry policy could use up a job's ceiling without a single model call behind it.

preflight takes the same idempotency_key now. The key is claimed inside the reserving transaction, so a duplicate waits on the unique index instead of racing: same key, same decision, one reservation. A retry that lands while the original is still being decided gets 409 preflight_in_progress, which isn't a refusal and reserves nothing.

Notice which way all of this fails. A reservation that never settles makes the ceiling tighter for an hour, never looser: the call that gets approved: false is a later one, until the sweeper gives the money back. Every choice above keeps that direction.

The test that runs on every change

We don't take the UPDATE's word for it. On every change, CI opens a job with a $1.00 ceiling and fires 40 preflights at it at once, each at the $0.10 default. It passes when exactly 10 are approved, 30 come back approved: false with task_ceiling_exceeded, the job's reserved_units is the ceiling to the micro-dollar, and the open reservation rows add up to the same number. A second job does it again at the caller's own $0.03 estimate and expects 33 approvals, $0.99, never 34.

40 preflights at once on a $1.00 job, $0.10 each

approved                                   10
approved: false (task_ceiling_exceeded)    30
task_budgets.reserved_units         1,000,000   = the ceiling
sum of open reservation rows        1,000,000

If someone changes that UPDATE into a read and a write, this is the test that goes red.

Using it

You don't write any of the SQL above. wrap() puts a preflight in front of each provider call and a record after it, so five parallel workers on one task_ref draw from one ceiling, in one process or in five.

Python

pip install -U agentbill-sdk openai

from concurrent.futures import ThreadPoolExecutor
from openai import OpenAI
from agentbill import wrap, Refusal

# Reads AGENTBILL_API_KEY; OpenAI() reads OPENAI_API_KEY.
# job-142 opens in dollars with a $5.00 ceiling.
llm = wrap(OpenAI(), task_ref="job-142", agent_id="researcher",
           task_ceiling_usd=5)

def look_up(topic):
    reply = llm.chat.completions.create(
        model="gpt-4o-mini",
        max_tokens=300,
        messages=[{"role": "user", "content": f"One line on {topic}."}],
    )
    if isinstance(reply, Refusal):
        # approved: false. This call was not sent. Your code decides.
        return None
    return reply.choices[0].message.content

topics = ["row locks", "idempotency keys", "reservations",
          "sweepers", "list prices"]
with ThreadPoolExecutor(max_workers=5) as pool:
    print(list(pool.map(look_up, topics)))

Node.js

npm install agentbill openai

import OpenAI from 'openai'
import { wrap, isRefusal } from 'agentbill'

// Reads AGENTBILL_API_KEY; new OpenAI() reads OPENAI_API_KEY.
// job-142 opens in dollars with a $5.00 ceiling.
const llm = wrap(new OpenAI(), {
  taskRef: 'job-142', agentId: 'researcher', taskCeilingUsd: 5,
})

const topics = ['row locks', 'idempotency keys', 'reservations',
                'sweepers', 'list prices']
const ask = (topic) => llm.chat.completions.create({
  model: 'gpt-4o-mini', max_tokens: 300,
  messages: [{ role: 'user', content: 'One line on ' + topic + '.' }],
})
const replies = await Promise.all(topics.map(ask))
for (const reply of replies) {
  // approved: false. This call was not sent. Your code decides.
  if (isRefusal(reply)) console.log(String(reply))
  else console.log(reply.choices[0].message.content)
}

AgentBill is an SDK inside your process, not a proxy. The call goes from your code to your provider, and AgentBill sees the provider's name, the model, the token counts and the job, never the prompt or the answer.