The Fundamentals course introduced batching as a loop: slice the list into 200s, post each slice, pause if anything was rate-limited. That loop is correct and it is enough for a few hundred rows. At production volume it has three gaps. It pays for duplicate inputs. It treats a chunk as all-or-nothing when the API reports success item by item. And after a pause it moves on to the next chunk, so the items that were turned away are simply lost.
This lesson builds the version you would run against a real list: plan the batches, send them, classify every item, and re-queue exactly the ones that are worth sending again.
The contract, precisely
A batch is per endpoint: POST https://api.apiverve.com/v1/<api>/batch with a JSON body. Every element of requests is the parameter object you would send to that endpoint on its own. GET and POST endpoints are written the same way, as keys on the object.
{ "requests": [ { "number": 42 }, { "number": 7 }, { "number": "abc" } ] }
The response:
{
"status": "ok",
"count": 3,
"billed": 2,
"rateLimited": 0,
"results": [
{ "status": "ok", "error": null, "data": { "number": 42, "words": "forty-two" } },
{ "status": "ok", "error": null, "data": { "number": 7, "words": "seven" } },
{ "status": "error", "error": "...", "data": null }
]
}
(data is trimmed here; each item carries the endpoint's full response.)
What each part guarantees:
| Part | Meaning |
|---|---|
| HTTP status | Whether the batch ran. 200 even if every item failed. A 4xx means the batch itself was refused: no requests array, more than 200 items, or a key problem that would stop a single call |
count | Items you sent |
billed | Items that cost credits. Multiply by the endpoint's per-call cost for credits spent |
rateLimited | Items turned away by this minute's rate window |
results[i] | The answer to requests[i]. Order is guaranteed, so the index is the correlation |
Each results entry is an ordinary envelope (status, error, data, and premium when fields were gated). Two kinds of item failure come from the batch machinery itself, and those carry a numeric code:
code: 429: the item did not fit in this minute's rate window. It also carriesretryAfter, in seconds.code: 402witherror: "insufficient credits": the account's balance ran out before this item.
Any other failed item has no code. It is the endpoint's own answer for that input (a validation error, nothing found), exactly what the same parameters would return as a single call. So branch on status first, and on code only to recognise the two batch-level refusals.
Two limits shape everything else: at most 200 items per call (201 is a 400 telling you to chunk), and one endpoint per batch. An empty requests array is legal and returns count: 0 immediately.
curl "https://api.apiverve.com/v1/numbertowords?number=200" \
-H "x-api-key: YOUR_API_KEY"One item's worth of work at 1 credit. A batch of 200 of these is still 200 credits; batching saves round trips, not credits.
How a batch spends credits and rate limit
Batching is easy to misread as a discount. It is not. Before running anything, the batch looks at two budgets and fills items in the order you sent them:
- Credits. It works out how many items your remaining balance can pay for at this endpoint's cost. Items past that point are answered with
402and never run. - Rate window. Of the affordable items, it takes as many as fit in your key's per-minute window and commits exactly that many slots. A batch of 150 uses 150 slots, not one. Items past the window come back as
429withretryAfter.
Then the runnable items execute, the bill is settled once for the whole batch, and the response carries x-api-remaining-credits with the balance after settlement, plus x-rate-limit-limit, x-rate-limit-remaining and, when something was turned away, retry-after. Those rate headers are readable from server code; browsers cannot see them across origins.
Consequences you design around:
- Chunk size is bounded by your rate limit, not just by 200. If your key's window holds 60 requests a minute, a 200-item batch runs 60 and returns 140
429s. Readx-rate-limit-limitfrom any response and size chunks to the room you actually have. - Credit-refused items do not burn rate slots. A
402item never reaches the window. - Order is priority. Credits and rate room are both handed out from the front of the array. If the list might exceed what you can afford, put the rows that matter most first.
- Only successful items bill. Validation failures,
402s and429s cost nothing.billed, notcount, is the number to reconcile. - Duplicates bill every time. Two identical items are two lookups and two charges. The API does not deduplicate for you.
That last point is the cheapest win in this lesson.
Your key's window allows 60 requests per minute and it is currently empty. You send one batch of 200 Email Validator items (20 credits each); 5 of the first 60 items are malformed emails, and the balance covers everything. What comes back?
Plan the batches: dedupe, then chunk
Deduplicate on the same canonical form you use for cache keys, so { a: 1, b: 2 } and { b: 2, a: 1 } are recognised as one lookup. Then chunk the unique items, and keep a map from each unique item back to every original row that asked for it, so results can be fanned out afterwards.
import { buildQuery } from './query.mjs';
export function planBatches(items, size = 200) {
if (!Number.isInteger(size) || size < 1) throw new RangeError('size must be a positive integer');
const seen = new Set();
const unique = [];
for (const item of items) {
const key = buildQuery(item);
if (seen.has(key)) continue;
seen.add(key);
unique.push(item);
}
const batches = [];
for (let i = 0; i < unique.length; i += size) batches.push(unique.slice(i, i + size));
return batches;
}
planBatches([{ a: 1, b: 2 }, { b: 2, a: 1 }, { a: 3 }], 2);
// [[{ a: 1, b: 2 }, { a: 3 }]]planBatches keeps the first occurrence of each distinct input, in the order it first appeared, so priority ordering survives deduplication. buildQuery from lesson 1 is the comparison: keys sorted, values encoded, null and undefined skipped. That last rule means { a: 1, b: null } and { a: 1 } count as the same input, which matches what the API does with an absent optional parameter.
When you need to map results back to rows, keep the key alongside each row:
import { buildQuery } from './query.mjs';
export function indexRows(rows, toParams) {
const byKey = new Map(); // key -> [row indexes]
rows.forEach((row, i) => {
const key = buildQuery(toParams(row));
if (!byKey.has(key)) byKey.set(key, []);
byKey.get(key).push(i);
});
return byKey;
}On a real customer list, deduplication is rarely a rounding error. Exports concatenated from several systems, the same company domain on every employee row, a currency pair repeated on every invoice line: 20 to 60 percent duplicates is common. On a 20-credit endpoint that is the difference between a job that fits this month's allowance and one that does not.
Normalise before deduplicating, with the same caution as for cache keys. Trimming whitespace from an email is safe. Lower-casing it merges Ada@Example.com and ada@example.com, which is right for nearly every mailbox in practice, but it is a decision, so make it deliberately and in one place.
Send, classify, re-queue
A robust sender treats every item as one of four outcomes:
| Item outcome | How you recognise it | What to do |
|---|---|---|
| Done | status === "ok" | Store data |
| Answered with an error | status === "error", no code | Store the error. It is that input's answer; sending it again returns the same thing and can trigger the repeated-failure throttle |
| Rate-limited | code === 429 | Re-queue after retryAfter seconds |
| Out of credits | code === 402 | Stop the job. Nothing else will succeed until the balance changes |
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
async function postBatch(api, requests) {
const res = await fetch(`https://api.apiverve.com/v1/${api}/batch`, {
method: 'POST',
headers: { 'x-api-key': process.env.APIVERVE_API_KEY, 'content-type': 'application/json' },
body: JSON.stringify({ requests }),
signal: AbortSignal.timeout(120_000),
});
const body = await res.json().catch(() => null);
if (!res.ok || body?.status !== 'ok') {
throw Object.assign(new Error(body?.error || `batch HTTP ${res.status}`), { status: res.status });
}
const limit = Number(res.headers.get('x-rate-limit-limit')) || null;
return { body, limit };
}
export async function runBatches(api, items, { size = 200, maxRounds = 10 } = {}) {
const results = new Map(); // index in items -> envelope
let queue = items.map((params, i) => ({ i, params }));
let chunk = size;
for (let round = 0; queue.length && round < maxRounds; round++) {
const retry = [];
let waitSeconds = 0;
for (let s = 0; s < queue.length; s += chunk) {
const slice = queue.slice(s, s + chunk);
const { body, limit } = await postBatch(api, slice.map((x) => x.params));
if (limit) chunk = Math.min(size, limit); // size to the real window
body.results.forEach((r, j) => {
const { i, params } = slice[j];
if (r.code === 402) throw Object.assign(new Error('insufficient credits'), { code: 402, done: results });
if (r.code === 429) { retry.push({ i, params }); waitSeconds = Math.max(waitSeconds, r.retryAfter ?? 60); return; }
results.set(i, r); // ok, or the input's own error
});
if (body.rateLimited > 0) break; // window full: stop sending this round
}
// Anything not yet sent this round goes back in the queue behind the retries.
const sent = new Set([...results.keys(), ...retry.map((x) => x.i)]);
queue = [...retry, ...queue.filter((x) => !sent.has(x.i))];
if (queue.length && waitSeconds) await sleep(waitSeconds * 1000 + Math.random() * 1000);
}
return { results, pending: queue };
}What the code is careful about:
- It throws only for batch-level failure. A non-
200or a non-okbatch status means the request itself was wrong; nothing inside it ran. Item errors never throw. - It re-queues only
429items, and puts them first. They were already at the front of your priority order, so they stay there. - It stops sending when the window is full. Posting the next chunk while
rateLimited > 0just converts that whole chunk into429s and keeps the key in the cooldown longer. It waits for the largestretryAfterseen, plus jitter so parallel workers do not return in lockstep. - It stops the job on
402and hands back what it has. The caller can store the completed results, and lesson 4 covers checking the budget before this ever happens. - It sizes chunks from
x-rate-limit-limit. After the first response it never sends more items than the window can hold. - The timeout is long. Nothing streams. The response arrives when the slowest item finishes, so a batch of a slow endpoint needs a deadline sized to that endpoint's latency, not to a single fast call. If you hit timeouts, make chunks smaller rather than the timeout longer.
maxRounds is the circuit breaker. A queue that never drains (a key capped at a very low per-minute limit, a stuck upstream) returns with pending set instead of looping for hours.
What not to retry
A batch timeout is the one ambiguous case. The batch may have run and billed on the server even though you never saw the response. Re-posting the whole chunk is safe for correctness, since APIVerve endpoints are lookups and transformations with no side effects to duplicate, but it can bill the chunk twice. Retry a timed-out chunk once, smaller, and count it in your cost reconciliation.
Never retry a 402 item: nothing changes until credits do. Never re-send a failed item that has no code: it is an answer, not a transient fault.
A batch response has HTTP 200, status: "ok", billed: 180 and rateLimited: 0. Twenty items have status: "error" with no code field. What should the job do with those twenty?
Reconcile what you paid
After the job, three numbers should agree:
- The sum of
billedacross batches, times the endpoint's credits per call. - The drop in
x-api-remaining-creditsbetween the first and last batch response (only exact if nothing else is spending on the account at the same time). - The count of
okresults you stored. On most endpoints this equalsbilled; when it does not, find out why before you scale the job up.
Log all three per run. When a run costs more than planned, the gap tells you where: billed higher than unique inputs means duplicates got through, a balance drop larger than billed times cost means something else shared the key, and stored results fewer than billed means your own code dropped answers you paid for.
Batching handles a list you can hold in memory and finish in minutes. The next lesson scales to lists you cannot, and to data that arrives a page at a time: Bulk jobs and pagination.