Retry with Exponential Backoff in JavaScript and Python

Networks fail in two ways: loudly and quietly. A loud failure is a refused connection or a 500, and your helper from the last two lessons already turns it into an exception. A quiet failure is a request that simply never finishes. Nothing throws, nothing logs, and the process sits there holding a connection until something upstream gives up. Retries have the opposite problem: written carelessly, they turn one failure into fifty and keep a rate limit going for minutes.

This lesson gives your helper a deadline, then teaches it to retry exactly the failures that a second attempt can fix.

Give every call a deadline

Neither fetch nor requests has a useful default timeout. Choose one per endpoint based on what it does. APIVerve's making requests guidance is 10 to 15 seconds for ordinary lookups and 30 seconds for rendering and extraction endpoints such as PDF or OCR, which do real work per call.

JavaScript: AbortSignal and AbortController

fetch accepts a signal. When the signal aborts, the request is cancelled and the Promise rejects. Node 18 and every current browser have a one-liner for a timer-driven signal:

const res = await fetch(url, {
  headers: { 'x-api-key': process.env.APIVERVE_API_KEY },
  signal: AbortSignal.timeout(10_000),
});

AbortSignal.timeout() is a shortcut for the longer form, which you still need when something other than a timer should cancel the request, such as a user navigating away or a newer search replacing an older one:

const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 10_000);
try {
  const res = await fetch(url, { headers, signal: controller.signal });
  // ...
} finally {
  clearTimeout(timer);
}

The finally matters. Without clearTimeout, the timer keeps your Node process alive for the full ten seconds after a fast response, and in a loop you accumulate timers.

A timed-out fetch rejects with an error whose name is TimeoutError (from AbortSignal.timeout) or AbortError (from controller.abort() with no reason). Check err.name, not the message text, which differs between runtimes.

Python: the timeout argument

requests takes timeout= on each call. A single number applies separately to connecting and to each wait between bytes of the response; a tuple sets them independently:

res = session.get(url, params=params, timeout=(3.05, 15))  # (connect, read)

A short connect timeout catches dead hosts quickly; a longer read timeout gives a slow endpoint room to answer. Note what the read timeout is not: it is not a cap on total duration. A server that trickles one byte every 14 seconds never trips it. If you need a hard wall-clock limit, enforce it around the call. Timeouts surface as requests.Timeout (with ConnectTimeout and ReadTimeout beneath it).

Check yourself

You call requests.get(url, timeout=15). Which statement is true?

Which failures are worth retrying

A retry is a bet that the next attempt will get a different answer. That only pays when the cause was temporary. Sort every failure into one of two buckets before writing any loop:

Failure	Retry?	Why
Network error, connection reset, timeout	Yes, with backoff	Transient by nature. The request may not have arrived at all.
`500`	Yes, with backoff	Something failed on the server side; it is usually gone on the next try.
`503`	Yes, with backoff	The upstream source was briefly unavailable.
`429` with credits left	Yes, after waiting	Per-minute rate limit. The same call would succeed a moment later.
`429` with `x-api-remaining-credits: 0`	Never	The monthly allowance is spent. Nothing changes until it renews or you upgrade.
`400`	No	Your input is wrong. The `error` sentence says what to change.
`401`	No	The key is not recognised. Retrying the same key gets the same answer.
`403` (JSON)	No	The key is valid but not permitted for this call: scoping, IP allow-list, or revoked.
`404`	No	No such endpoint, or nothing found for that input (the dictionary returns `404` for an unknown word).

The pattern underneath is simple: retry what is wrong with the moment, never what is wrong with the request. A 4xx is a statement about your request. Sending it again unchanged is not persistence; it is a loop. APIVerve treats it as one, too: when the same call keeps failing the same way, that exact call is throttled and the original error is replayed back, so the retries do not even reach the API.

The 429 that looks like the other 429

APIVerve uses 429 for two different situations, and they need opposite responses:

{ "status": "error", "error": "Rate limit exceeded. Max 60 requests per minute.", "data": null }

{ "status": "error", "error": "Monthly credit limit reached. Credits will reset on your renewal date.", "data": null }

The first is "slow down". The second is "the month is spent". Do not tell them apart by matching the message: there are several wordings for the credit case. Read the x-api-remaining-credits response header. If it is 0, stop and tell a human. Otherwise wait and retry, honouring the retry-after header when it is present.

There is a classic bug in that test. Number(res.headers.get('x-api-remaining-credits')) === 0 looks right, but headers.get returns null for a missing header and Number(null) is 0. Your code would conclude it is out of credits whenever the header is absent. Check for null first.

Check yourself

A call returns 429. The x-api-remaining-credits header reads 0. What should your client do?

Answer the check above to continue.

Exponential backoff with jitter

Once you know a failure is worth retrying, the next question is how long to wait. Retrying immediately is the worst option: if the server is overloaded, you add load at the exact moment it can least take it. A fixed delay is better but still synchronises clients: every caller that failed at the same instant retries at the same instant.

Exponential backoff doubles the wait after each failure: 0.5 s, 1 s, 2 s, 4 s. A cap stops it growing absurd. Jitter randomises each wait so that a crowd of clients spreads out instead of arriving in waves. The common form, "full jitter", picks a random delay between zero and the exponential ceiling:

delay = random(0, min(cap, base * 2 ** attempt))

If the server sent retry-after, use that instead. It knows more about its own state than your formula does. Add a little jitter on top so that clients told to wait the same number of seconds do not all return together.

Putting it into the helper

Here is the JavaScript client with all of the above. It keeps the shape of lesson one's apiverve() and adds a timeout, a classification of every failure, and a bounded retry loop. ES modules, Node 18+.

apiverve-retry.mjs

const BASE = 'https://api.apiverve.com/v1';

export class ApiError extends Error {
  constructor(message, { status = 0, retryable = false, retryAfter = null } = {}) {
    super(message);
    this.name = 'ApiError';
    this.status = status;          // 0 means no HTTP response at all
    this.retryable = retryable;
    this.retryAfter = retryAfter;  // seconds, when the server said
  }
}

async function attempt(id, params, timeoutMs) {
  const key = process.env.APIVERVE_API_KEY;
  if (!key) throw new Error('APIVERVE_API_KEY is not set');

  const url = new URL(`${BASE}/${id}`);
  for (const [k, v] of Object.entries(params)) url.searchParams.set(k, String(v));

  let res;
  try {
    res = await fetch(url, {
      headers: { 'x-api-key': key },
      signal: AbortSignal.timeout(timeoutMs),
    });
  } catch (err) {
    const timedOut = err.name === 'TimeoutError' || err.name === 'AbortError';
    throw new ApiError(timedOut ? `Timed out after ${timeoutMs} ms` : `Network error: ${err.message}`, {
      retryable: true,
    });
  }

  let body = null;
  try { body = await res.json(); } catch {}

  if (res.ok && body?.status === 'ok') return body.data;

  const message = body?.error || `HTTP ${res.status}`;

  if (res.status === 429) {
    const credits = res.headers.get('x-api-remaining-credits');
    if (credits !== null && Number(credits) === 0) {
      throw new ApiError(`Out of credits: ${message}`, { status: 429 });
    }
    const ra = Number(res.headers.get('retry-after'));
    throw new ApiError(message, { status: 429, retryable: true, retryAfter: ra > 0 ? ra : null });
  }

  throw new ApiError(message, { status: res.status, retryable: res.status >= 500 });
}

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

export async function apiverve(id, params = {}, { tries = 4, timeoutMs = 10_000, baseMs = 500, capMs = 8_000 } = {}) {
  for (let i = 0; ; i++) {
    try {
      return await attempt(id, params, timeoutMs);
    } catch (err) {
      const last = i >= tries - 1;
      if (!(err instanceof ApiError) || !err.retryable || last) throw err;

      const wait = err.retryAfter
        ? err.retryAfter * 1000 + Math.random() * 500
        : Math.random() * Math.min(capMs, baseMs * 2 ** i);
      await sleep(wait);
    }
  }
}

Read the control flow once more with the table in mind. A 400, 401, 403 or 404 throws with retryable: false, so the loop rethrows on the first failure. Credit exhaustion throws non-retryable before any waiting happens. Only network errors, timeouts, 5xx and a rate-limit 429 reach the sleep. And tries bounds the whole thing, so even a retryable failure eventually becomes an error someone sees.

Try it against an endpoint whose answer changes every time you call it, which makes it easy to confirm you are not looking at a stale result:

Try it liveWorld Time2 credits

GEThttps://api.apiverve.com/v1/worldtime

1
2

curl "https://api.apiverve.com/v1/worldtime?city=Tokyo" \
  -H "x-api-key: YOUR_API_KEY"

city

data.foundCities[0].time24 changes on every call. Compare it with your own client's output.

use-retry.mjs

import { apiverve, ApiError } from './apiverve-retry.mjs';

try {
  const data = await apiverve('worldtime', { city: 'Tokyo' });
  console.log(data.foundCities[0].timezone, data.foundCities[0].time24);
} catch (err) {
  if (err instanceof ApiError && err.status === 429 && !err.retryable) {
    console.error('Out of credits. Stop the job and alert someone.');
  } else {
    throw err;
  }
}

The same logic in Python

apiverve_retry.py

import os
import random
import time
import requests

BASE = "https://api.apiverve.com/v1"
_session = requests.Session()
_session.headers["x-api-key"] = os.environ["APIVERVE_API_KEY"]


class ApiError(Exception):
    def __init__(self, message, status=0, retryable=False, retry_after=None):
        super().__init__(message)
        self.status = status
        self.retryable = retryable
        self.retry_after = retry_after


def _attempt(api_id, params, timeout):
    try:
        res = _session.get(f"{BASE}/{api_id}", params=params, timeout=timeout)
    except (requests.ConnectionError, requests.Timeout) as err:
        raise ApiError(f"Network error: {err}", retryable=True) from err

    try:
        body = res.json()
    except ValueError:
        body = None
    if res.ok and isinstance(body, dict) and body.get("status") == "ok":
        return body["data"]

    message = (body or {}).get("error") or f"HTTP {res.status_code}"

    if res.status_code == 429:
        credits = res.headers.get("x-api-remaining-credits")
        if credits is not None and float(credits) == 0:
            raise ApiError(f"Out of credits: {message}", status=429)
        try:
            retry_after = float(res.headers.get("retry-after", ""))
        except ValueError:
            retry_after = None
        raise ApiError(message, status=429, retryable=True, retry_after=retry_after)

    raise ApiError(message, status=res.status_code, retryable=res.status_code >= 500)


def apiverve(api_id, params=None, tries=4, timeout=(3.05, 10), base=0.5, cap=8.0):
    for i in range(tries):
        try:
            return _attempt(api_id, params or {}, timeout)
        except ApiError as err:
            if not err.retryable or i == tries - 1:
                raise
            if err.retry_after:
                delay = err.retry_after + random.uniform(0, 0.5)
            else:
                delay = random.uniform(0, min(cap, base * 2 ** i))
            time.sleep(delay)

Retrying POST safely

Everything above used GET, which is read-only by definition: sending it twice cannot change anything on the server. POST is where retries get dangerous in general. If a payment API times out after it charged the card but before you got the response, a blind retry charges it twice. Your client cannot know which side of the failure it was on.

APIs that create things solve this with an idempotency key: you generate a unique id per logical operation, send it as a header, and reuse it on every retry. The server recognises the repeat and returns the original result instead of acting twice. When an API offers one, always send it on retried POSTs.

APIVerve's POST endpoints are transformations rather than record creation: sending the same body twice gives the same answer and creates nothing twice. There is no idempotency key to manage. The only cost of a duplicate is the credits, which is one more reason the retry rules above aim at avoiding pointless attempts rather than adding more of them.

What you have now

Your helper has a deadline on every call, a clear rule for what gets retried, waits that grow and spread out, and a hard stop on the one 429 that waiting cannot fix. The remaining weak point is not in the network at all: it is the key itself, and where it ends up. Next: Keep API keys secret.

Finished the lesson?Mark it complete to track your progress toward the certification.

Next: Keep API keys secret →

Timeouts, retries and backoff