Cache API Responses in JavaScript and Python to Save Credits

Up to now every call has been treated as free: write the helper, call it whenever you need an answer. On a metered API that habit is how a weekend side project burns a month's allowance on Saturday night. A page that looks up the visitor's country on every load, a loop that validates the same email list each time the script runs, a retry that fires on a credit-exhaustion error: each one is cheap per call and expensive in aggregate.

This lesson is about the arithmetic. Know what a call costs, avoid making the same call twice, and check the balance before committing to a big job.

What a call costs

On APIVerve each API has a fixed credit cost per successful call, set in its definition and shown on its reference page. Most cost one or two credits; APIs that do more work per request cost more. The APIs used in this course:

API	Credits per call	What drives the cost
Number to Words	1	Pure computation
Country Lookup	2	Reference data lookup
Exchange Rate	2	Live rates
Dictionary, World Time	2	Lookup
IP Lookup	10	Geolocation database
Email Validator	20	Several live DNS checks per address

The spread is the point. Ten thousand page views that each call Number to Words cost 10,000 credits. The same traffic calling Email Validator costs 200,000. Before you put any call in a hot path, multiply its cost by how often that path runs.

Two billing rules change how you write error handling:

Failed calls are not billed. A 400, a rejected key or a blocked call costs nothing, because it never reached the source that charges.
An empty successful answer is billed. A lookup that ran and found nothing did the work you asked for. "No result" is still a result.

Neither rule makes retrying a 4xx free in practice: it still costs you time, rate-limit headroom, and eventually the repeated-failure throttle from lesson three.

Cache what does not change

A cached response costs zero credits and answers in microseconds. The API does not cache for you, and it sends no Cache-Control header that would let a proxy do it, because a cached answer you did not ask for is a wrong answer on the endpoints that change by the second. Caching is a decision you make per endpoint, based on how fast the underlying data actually moves.

Kind of data	Reasonable cache lifetime	Example
Country, currency, timezone, ISO codes	Weeks	Country Lookup for `JP`
Pure functions of the input	Indefinitely	Number to Words for `1999`
Geocoding, DNS, WHOIS, company records	Hours to a day	IP Lookup
Weather, exchange rates, market data	Minutes	Exchange Rate `USD` to `EUR`
Validation, current time, random output	Do not cache	Email Validator, World Time

The last row deserves a concrete example. Validating [email protected] twice a few minutes apart returned the same verdict but different mxRecord values (mxa-... and then mxb-...), because the domain publishes several mail servers and DNS rotates through them. A validation result describes the world at the moment of the call. Caching it for a day means trusting yesterday's DNS. And anything time-based or random is wrong the instant it is cached.

Try it liveCountry Lookup2 credits

GEThttps://api.apiverve.com/v1/countrylookup

1
2

curl "https://api.apiverve.com/v1/countrylookup?country=JP" \
  -H "x-api-key: YOUR_API_KEY"

country

Nothing in data.countries[0] will change for years. This is the ideal thing to cache.

A TTL cache in JavaScript

A Map with an expiry time per entry is enough for a single process. This builds on the apiverve() helper from lesson three.

cache.mjs

import { apiverve } from './apiverve-retry.mjs';

const cache = new Map();
const MAX_ENTRIES = 1000;

function cacheKey(id, params) {
  const sorted = Object.keys(params).sort().map((k) => [k, String(params[k])]);
  return `${id}?${new URLSearchParams(sorted)}`;
}

export async function cached(id, params, ttlMs) {
  const key = cacheKey(id, params);
  const hit = cache.get(key);
  if (hit && hit.expires > Date.now()) return hit.value;

  const value = apiverve(id, params);           // store the Promise itself
  cache.set(key, { value, expires: Date.now() + ttlMs });
  if (cache.size > MAX_ENTRIES) cache.delete(cache.keys().next().value);

  try {
    return await value;
  } catch (err) {
    cache.delete(key);                          // never cache a failure
    throw err;
  }
}

Usage:

const WEEK = 7 * 24 * 60 * 60 * 1000;
const a = await cached('countrylookup', { country: 'JP' }, WEEK);  // 2 credits
const b = await cached('countrylookup', { country: 'JP' }, WEEK);  // 0 credits
console.log(a.countries[0].capital[0]);                            // "Tokyo"

Each line in cached() is there to prevent a specific bug:

The key is built from sorted parameters. {from, to} and {to, from} are the same question and should share one entry. Without sorting, you pay twice for an identical lookup.
The Promise is cached, not the resolved value. If twenty requests for JP arrive in the same millisecond on a cold cache, they all find the same pending Promise and share one API call. Caching only after the await would fire twenty calls and pay forty credits. This is called a cache stampede, and it happens on every server restart.
Failures are removed. A cached rejection would replay a transient 503 to every caller until the entry expired.
The size is capped. A Map iterates in insertion order, so deleting the first key evicts the oldest entry. Without a cap, a cache keyed on user input is a memory leak with extra steps.

Normalise inputs before they reach the key when the API treats them the same, such as trimming whitespace or upper-casing currency codes. Do it only when you are sure the API really does treat them the same; normalising away a real difference serves one user another user's answer.

Caching in Python

For pure lookups, the standard library already has a cache:

lookups.py

from functools import lru_cache
from apiverve_retry import apiverve


@lru_cache(maxsize=512)
def number_words(n):
    return apiverve("numbertowords", {"number": n})["words"]


print(number_words(1999))   # calls the API
print(number_words(1999))   # served from memory
print(number_words.cache_info())

lru_cache is the right tool for exactly this case and the wrong one for most others. It keys on the function's arguments, so they must be hashable: you cannot pass a params dict, which is why the wrapper takes a single number. It has no expiry, so it suits data that never changes for the life of the process. It does not cache exceptions, which is good. And it hands every caller the same object: if one caller mutates a returned dict, every later caller sees the mutation. Returning a string, as above, avoids that.

When data does go stale, use a dict with timestamps:

ttl_cache.py

import time
from apiverve_retry import apiverve

_cache = {}


def cached(api_id, params, ttl_seconds):
    key = (api_id, tuple(sorted((k, str(v)) for k, v in params.items())))
    hit = _cache.get(key)
    if hit and hit[1] > time.monotonic():
        return hit[0]
    data = apiverve(api_id, params)       # raises on failure, so nothing is cached
    _cache[key] = (data, time.monotonic() + ttl_seconds)
    return data


rate = cached("exchangerate", {"currency1": "USD", "currency2": "EUR"}, ttl_seconds=300)
print(float(rate["exchangeRate"]))

Use time.monotonic() rather than time.time() for expiry: the wall clock can jump when the system syncs its time, and a backwards jump would keep stale entries alive. Note the float() on exchangeRate: the API returns rates as strings to preserve precision, and string arithmetic in Python fails loudly where JavaScript would coerce quietly.

Try it liveExchange Rate2 credits

GEThttps://api.apiverve.com/v1/exchangerate

1
2

curl "https://api.apiverve.com/v1/exchangerate?currency1=USD&currency2=EUR" \
  -H "x-api-key: YOUR_API_KEY"

currency1currency2

data.exchangeRate is a string, and data.lastUpdated shows how fresh it is. Five minutes of caching is reasonable.

Check yourself

Twenty requests for the same uncached country arrive at once. Your cache stores the resolved data only after await apiverve(...) finishes. How many API calls are made?

Read the balance from every response

You do not need a separate call to know how many credits you have. Every response carries the account's position in headers:

Header	Meaning
`x-api-remaining-credits`	Credits left this cycle
`x-api-credits-used`	Credits spent this cycle
`x-api-max-credits`	The cycle's allowance
`x-api-renewal`	Unix timestamp when the allowance resets

The first three are readable from browser JavaScript as well as from a server, so a dashboard page can show a meter without extra calls. Your helper currently throws headers away, so give it a variant that keeps them:

meter.mjs

export async function apiverveWithCredits(id, params = {}) {
  const url = new URL(`https://api.apiverve.com/v1/${id}`);
  for (const [k, v] of Object.entries(params)) url.searchParams.set(k, String(v));

  const res = await fetch(url, {
    headers: { 'x-api-key': process.env.APIVERVE_API_KEY },
    signal: AbortSignal.timeout(10_000),
  });
  const body = await res.json().catch(() => null);
  const left = res.headers.get('x-api-remaining-credits');
  const remaining = left === null ? null : Number(left);

  if (!res.ok || body?.status !== 'ok') {
    throw Object.assign(new Error(body?.error || `HTTP ${res.status}`), { status: res.status, remaining });
  }
  return { data: body.data, remaining };
}

A budget check before a bulk job

The useful thing to do with that number is refuse to start a job you cannot finish. Before validating a list of addresses, one cheap call tells you the balance, and the arithmetic tells you whether the job fits:

budget.mjs

import { apiverveWithCredits } from './meter.mjs';

const COST = { emailvalidator: 20, numbertowords: 1 };   // from each API's reference page

export async function assertBudget(id, count, { reserve = 500 } = {}) {
  const { remaining } = await apiverveWithCredits('numbertowords', { number: 1 }); // 1-credit probe
  const needed = COST[id] * count;
  if (remaining !== null && remaining - needed < reserve) {
    throw new Error(`Need ${needed} credits for ${count} ${id} calls, have ${remaining} (keeping ${reserve} in reserve)`);
  }
  return remaining;
}

The reserve is the judgment call. A nightly job that happily spends the last credit takes your production signup form down with it, because the form's validator now gets an out-of-credits 429 too. Keep a floor for the traffic that matters, and let the batch job be the one that waits for renewal.

Batching: fewer requests, same credits

When you have a list of inputs for one endpoint, APIVerve's batch requests send up to 200 of them in a single POST to /v1/<id>/batch. Be clear about what that does and does not buy:

It saves round trips, not rate limit. A batch is one HTTP request with one authentication, but it is not a way around the per-minute limit: it takes a slot in the minute's window for each item it runs, and items that do not fit come back as per-item 429s, counted in the response's rateLimited field.
It does not save credits. Each item is billed as if called alone, at the endpoint's normal cost. Items that fail validation are not billed, so the response's billed count can be lower than count.
It can run out partway. A batch larger than your remaining balance runs as far as the credits go; later items come back individually with insufficient credits and a 402 code. Put the inputs you care most about first.

batch.mjs

const CHUNK = 200;

export async function validateEmails(emails) {
  const results = [];
  for (let i = 0; i < emails.length; i += CHUNK) {
    const requests = emails.slice(i, i + CHUNK).map((email) => ({ email }));
    const res = await fetch('https://api.apiverve.com/v1/emailvalidator/batch', {
      method: 'POST',
      headers: {
        'x-api-key': process.env.APIVERVE_API_KEY,
        'content-type': 'application/json',
      },
      body: JSON.stringify({ requests }),
      signal: AbortSignal.timeout(60_000),
    });
    const body = await res.json().catch(() => null);
    if (!res.ok || body?.status !== 'ok') throw new Error(body?.error || `HTTP ${res.status}`);

    results.push(...body.results);   // same order as `requests`
    if (body.results.some((r) => r.code === 402)) break;   // out of credits: stop sending chunks
    if (body.rateLimited > 0) await new Promise((r) => setTimeout(r, 60_000));
  }
  return results;
}

Every element of results is an ordinary envelope, so check each item's status; the batch's 200 only says the batch ran. Results come back in the order you sent them, so results[3] answers requests[3] with no id matching needed. The timeout is longer than for a single call because nothing streams: the response arrives when the slowest item finishes.

Batching is worth it for a column of values, a nightly re-check of a stored list, or enriching an import. It is not worth it for one lookup in a request path, and it never combines different endpoints in one call.

Check yourself

You batch 150 Email Validator lookups (20 credits each) in one call instead of 150 separate calls, and 10 of the inputs are malformed. Roughly what do you pay?

Answer the check above to continue.

Caching and credit budgets

What a call costs

Cache what does not change

A TTL cache in JavaScript

Caching in Python

Read the balance from every response

A budget check before a bulk job

Batching: fewer requests, same credits

The course in one helper

Run every example on your own key. Free plan, no card.

Scaling up?