Lesson two made retries safe. This lesson is about needing fewer of them. At scale, almost every 429 you receive is self-inflicted: workers that each think they own the whole limit, a backfill sharing a key with live traffic, a burst at the top of every minute. And almost every credit outage is predictable days in advance from numbers that arrive on every response.
The goal is a client that never trips the per-minute limit in normal operation, reacts to the one 429 that slips through by slowing everyone down, and stops cleanly, with an alert already sent, when credits run out.
Know the ceilings you are pacing to
APIVerve enforces a per-minute request limit per API key, set by the plan. The rate limits page renders the current table from the published plans file. At the time of writing:
| Plan | Requests per minute | Concurrent calls (published) |
|---|---|---|
| Free | 5 | 1 |
| Starter | 60 | 5 |
| Pro | 180 | 20 |
| Mega | No per-minute limit | 50 |
Four details change how you design around those numbers.
- The window slides. The limiter counts your requests in the last 60 seconds, not per clock minute. On a rate-limit
429,retry-afteris the number of seconds until the oldest request in that window ages out. Sixty calls fired in the first second of a minute lock you out for the next 59; sixty calls spread evenly never trip it. - The effective limit can be lower than the plan's. A key can be capped below its plan in the dashboard, and the lowest applicable cap wins. Server-side,
x-rate-limit-limiton a response reports the limit actually in force andx-rate-limit-remainingwhat is left in the window. Neither is readable from browser JavaScript, because only the four credit headers are exposed cross-origin. - Sub-keys have their own window, but share the account's credits. A sub-key is rate limited separately from the main key and from other sub-keys, and can carry its own lower limit. Credits are billed to the account. So sub-keys isolate pace, not budget.
- "No per-minute limit" is not unlimited. Mega has no per-minute ceiling, but the credit allowance and the published concurrency ceiling still apply, and the edge has a coarse flood limit for every key.
What happens when you overrun
The 429 itself is the smaller cost. When the origin rate-limits a key, the edge starts holding that key in a cooldown for the rest of the window (the rate limits page describes it as around 50 seconds). During the cooldown the edge answers 429 itself, with retry-after set to the seconds remaining, without forwarding anything. Retrying during the cooldown does not shorten it.
The cooldown applies to the key, not to the worker that caused it. On a shared key, one worker that overruns the window pauses every other worker using that key, including your production traffic. This is the strongest argument for giving each workload its own sub-key: a backfill that overruns its own window stalls only itself.
A separate guard covers loops. When the same request keeps failing the same way, the edge throttles that exact request and replays the original error, so a stuck retry loop on a bad input does not reach the API at all. It is not a rate limit you should design around; it is a sign the loop should have stopped already.
A token bucket sized to the window
A token bucket is the standard client-side pacer. Tokens drip in at a fixed rate up to a capacity, and every call takes one. The capacity allows a small burst; the rate sets the long-run pace.
Sizing it against a sliding 60-second window has a precise rule. In any 60-second span the bucket can release at most its full capacity plus everything that refilled during those 60 seconds:
capacity + ratePerSecond * 60 <= limitPerMinuteFor a Starter key at 60 per minute, a capacity of 6 leaves room for a rate of (60 - 6) / 60 = 0.9 calls per second. That guarantees you can never exceed the window from this bucket, whatever your call pattern. Leave a further margin if anything else uses the same key.
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
export class TokenBucket {
constructor({ ratePerSecond, capacity }) {
Object.assign(this, { ratePerSecond, capacity });
this.tokens = capacity;
this.last = Date.now();
this.pausedUntil = 0;
}
static forLimit(limitPerMinute, capacity = Math.max(1, Math.floor(limitPerMinute / 10))) {
return new TokenBucket({ capacity, ratePerSecond: (limitPerMinute - capacity) / 60 });
}
#refill(now) {
this.tokens = Math.min(this.capacity, this.tokens + ((now - this.last) / 1000) * this.ratePerSecond);
this.last = now;
}
async take() {
for (;;) {
const now = Date.now();
if (now < this.pausedUntil) { await sleep(this.pausedUntil - now); continue; }
this.#refill(now);
if (this.tokens >= 1) { this.tokens -= 1; return; }
await sleep(((1 - this.tokens) / this.ratePerSecond) * 1000);
}
}
pause(ms) { // a 429 slipped through: everyone waits
this.pausedUntil = Math.max(this.pausedUntil, Date.now() + ms);
this.tokens = 0;
}
}The pause() method is what turns a 429 into a fleet-wide slowdown instead of one worker's private problem. When any call sees a rate-limit 429, it calls bucket.pause(retryAfterMs) and every worker waiting on take() waits too, which is exactly what the edge cooldown is going to enforce anyway.
Concurrency is a separate ceiling
A rate limit is about how many calls start per minute. Concurrency is how many are in flight at once. Slow endpoints (rendering, OCR, crawling) can hit the concurrency ceiling while far under the rate limit. Run a fixed pool of workers matched to your plan's published concurrency, and take a token before each call:
import { TokenBucket } from './token-bucket.mjs';
import { resilientCall } from './resilient-call.mjs';
import { RateLimitError, CreditsExhaustedError } from './errors.mjs';
import { parseRetryAfter } from './retry-after.mjs';
export async function runPool(items, { api, limitPerMinute = 60, concurrency = 5, toParams }) {
const bucket = TokenBucket.forLimit(limitPerMinute);
const results = new Array(items.length);
const again = []; // items to resend after a 429
let next = 0;
let halted = null;
async function worker() {
while (!halted && (again.length || next < items.length)) {
const i = again.length ? again.shift() : next++;
await bucket.take();
try {
results[i] = { ok: true, data: await resilientCall(api, toParams(items[i]), { maxAttempts: 1 }) };
} catch (err) {
if (err instanceof RateLimitError) {
bucket.pause(parseRetryAfter(err.retryAfter, Date.now()) ?? 30_000);
again.push(i); // resend it once the pause is over
continue;
}
if (err instanceof CreditsExhaustedError) { halted = err; break; }
results[i] = { ok: false, error: err };
}
}
}
await Promise.all(Array.from({ length: concurrency }, worker));
return { results, halted };
}Here the pool owns rate-limit handling (maxAttempts: 1 turns off the per-call retry so the shared bucket decides when to resend), while transient 5xx failures are recorded for a later pass. In a long-running service you would let resilientCall retry 5xx itself and keep only the 429 path in the pool.
A Starter key allows 60 requests per minute on a sliding window. Your token bucket has capacity 10. What is the highest refill rate that can never exceed the limit in any 60-second span?
One limit, many processes
A bucket in memory only paces its own process. Run four replicas, each with TokenBucket.forLimit(60), and the key sees up to 240 calls a minute. There are three ways out, in increasing order of effort.
- Split the limit statically. Give each of N processes
limit / N. Simple and correct, but it wastes capacity whenever replicas are idle, and it breaks silently when autoscaling changes N. Good enough for a fixed number of cron workers. - One key per workload, one dispatcher per key. Put all calls for a key through a single consumer process that owns the bucket. Throughput is capped by one process, which is rarely the bottleneck at these limits.
- A shared pacer in Redis. Every process asks a central store for the next free slot, and waits until then.
The third option fits in a short Lua script. Redis runs scripts atomically, so two workers can never be handed the same slot, and using Redis's own clock (TIME) avoids clock skew between machines.
import Redis from 'ioredis';
const redis = new Redis(process.env.REDIS_URL);
// Hand out the next slot at least intervalMs after the previous one. Returns ms to wait.
redis.defineCommand('reserveSlot', {
numberOfKeys: 1,
lua: `
local t = redis.call('TIME')
local now = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)
local nextFree = tonumber(redis.call('GET', KEYS[1]) or '0')
local slot = math.max(nextFree, now)
redis.call('SET', KEYS[1], slot + tonumber(ARGV[1]), 'PX', 120000)
return slot - now`,
});
// Push the next free slot out to at least now + pauseMs (after a 429).
redis.defineCommand('pauseSlots', {
numberOfKeys: 1,
lua: `
local t = redis.call('TIME')
local now = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)
local nextFree = tonumber(redis.call('GET', KEYS[1]) or '0')
redis.call('SET', KEYS[1], math.max(nextFree, now + tonumber(ARGV[1])), 'PX', 120000)
return 1`,
});
export function sharedPacer(name, limitPerMinute, headroom = 0.9) {
const key = `pace:${name}`;
const intervalMs = Math.ceil(60_000 / (limitPerMinute * headroom));
return {
async take() {
const waitMs = await redis.reserveSlot(key, intervalMs);
if (waitMs > 0) await new Promise((r) => setTimeout(r, waitMs));
},
pause: (ms) => redis.pauseSlots(key, ms),
};
}This is a leaky bucket rather than a token bucket: it never bursts, it spaces every call exactly intervalMs apart across the whole fleet. For a Starter key with 10% headroom that is one call every 1,112 ms from all processes combined. Even spacing is the shape a sliding window rewards, so the lack of burst costs nothing here. Two cautions: a worker that reserves a slot far in the future has committed to it, so check the wait against your deadline and give up early rather than sleeping past it; and keep headroom below 1, because your clock and the platform's window never line up exactly, and a fleet paced at exactly 100% will see the occasional 429.
The same pause() contract from the in-memory bucket applies. Any process that receives a rate-limit 429 pushes the shared schedule out by retry-after, and every process obeys it on its next take().
Read the credit headers on every response
Every metered response tells you where the account stands:
| Header | Meaning |
|---|---|
x-api-remaining-credits | Credits left this billing cycle |
x-api-credits-used | Credits spent this cycle |
x-api-max-credits | The cycle's allowance |
x-api-renewal | Unix timestamp (seconds) when the allowance resets |
These count credits, not calls. Each API has its own cost per successful call, listed on its reference page, so a call to a 1-credit API and a call to a 20-credit API move the meter by different amounts. A batch response carries x-api-remaining-credits as it stands after the batch is settled. Run a cheap call in your own client and watch the headers move:
curl "https://api.apiverve.com/v1/numbertowords?number=60" \
-H "x-api-key: YOUR_API_KEY"The response body shows the data. In your own server-side client, read x-api-remaining-credits before and after: it drops by this API's credit cost.
Feed every response's headers into one meter per account. Ignore responses that do not carry x-api-max-credits; a response without it was not metered and says nothing about the balance.
Alert before it happens
A threshold alert ("80% used") is better than nothing, but it fires at the same point whether you are on day 3 of the cycle or day 29. What you actually want to know is: at the current burn rate, will credits run out before renewal?
export class CreditMeter {
constructor({ alert, windowMs = 6 * 3600_000, thresholds = [0.5, 0.8, 0.95] }) {
Object.assign(this, { alert, windowMs, thresholds });
this.samples = []; // [timeMs, used]
this.fired = new Set(); // alerts already sent this cycle
this.state = null;
}
observe(res, now = Date.now()) {
const max = res.headers.get('x-api-max-credits');
const used = res.headers.get('x-api-credits-used');
const remaining = res.headers.get('x-api-remaining-credits');
if (max === null || used === null || remaining === null) return; // not metered
const renewal = Number(res.headers.get('x-api-renewal')) || null;
if (this.state && renewal && this.state.renewal && renewal !== this.state.renewal) {
this.samples = []; this.fired.clear(); // new cycle
}
this.state = { max: Number(max), used: Number(used), remaining: Number(remaining), renewal };
this.samples.push([now, Number(used)]);
this.samples = this.samples.filter(([t]) => now - t <= this.windowMs);
this.#check(now);
}
#check(now) {
const { max, used, remaining, renewal } = this.state;
for (const th of this.thresholds) {
if (used / max >= th) this.#once(`pct-${th}`, `${Math.round(th * 100)}% of credits used (${used}/${max})`);
}
const [t0, u0] = this.samples[0];
const hours = (now - t0) / 3600_000;
if (hours < 0.5 || !renewal) return; // not enough data to project
const perHour = (used - u0) / hours;
if (perHour <= 0) return;
const emptyAt = now + (remaining / perHour) * 3600_000;
if (emptyAt < renewal * 1000) {
const days = ((renewal * 1000 - emptyAt) / 86_400_000).toFixed(1);
this.#once('projected', `At ${Math.round(perHour)} credits/hour, credits run out ${days} days before renewal`);
}
}
#once(id, message) {
if (this.fired.has(id)) return;
this.fired.add(id);
this.alert(message);
}
}The projection uses a trailing window (six hours here) so that a nightly batch does not dominate the estimate for the whole day, and it refuses to project from less than half an hour of data. Each alert fires once per cycle. Tune the window to your traffic: a service with a strong daily cycle projects better from a 24-hour window.
x-api-remaining-credits reads 40,000, renewal is 10 days away, and the meter has measured a steady 250 credits per hour over the last six hours. What should the projection alert say?
Stop cleanly when credits run out
When a call fails with CreditsExhaustedError (a 429 whose credit header reads zero or less, or a batch item with code: 402), every later call on the account will fail the same way until renewal or a top-up. A clean stop means:
- Stop taking new work. Set a process-wide halted flag that workers check before pulling the next message.
- Do not mark the work as failed. Messages that hit exhaustion should go back to the queue (or simply not be acknowledged), not into a dead-letter queue. Nothing is wrong with them.
- Alert once, as a billing event, to whoever can top up or upgrade. Include the renewal date from
x-api-renewalso they know how long the outage lasts if they do nothing. - Probe for recovery slowly. One cheap call every few minutes is enough. After a top-up, the edge, which remembers an exhausted key and answers for it directly, notices the new balance on its next periodic check, so recovery takes seconds, not minutes. Probing faster than that only adds noise.
- Resume from where you stopped. If the job is idempotent (the next lesson), resuming is simply restarting the consumers.
Combine this with the reserve from the Fundamentals course: a bulk job that refuses to start, or halts itself, when the projected balance would cross a floor you keep for production traffic. The best credit outage is the one your batch job causes to itself while your signup form keeps working.
What you have now
Your fleet now paces itself to the key's window with a correctly sized bucket, shares one pace across processes, treats a single 429 as a signal for everyone, keeps workloads on separate sub-keys so one cannot stall another, forecasts credit exhaustion days ahead and stops cleanly when it happens anyway. The last piece is making all of this safe to restart: a halted job, a redelivered message or a replayed batch must never do the same side effect twice. Next: Idempotency and safe replays.