Every integration test suite eventually faces the same dilemma. Tests against the live API are slow, cost credits, and fail when the network does. Tests against hand-written mocks are fast and free, and they pass happily while production breaks, because the mock encodes what you believed the API returns, not what it actually returns. The usual outcome is a suite that is either expensive and flaky or cheap and unconvincing.
The way out is to stop treating "the tests" as one thing. This lesson builds four layers, each answering a different question at a different cost: pure-function units, recorded fixtures, live contract tests, and post-deploy smoke tests. Then it bounds what the live layers can spend.
The integration test pyramid
| Layer | Question it answers | Talks to the API? | Runs | Cost |
|---|---|---|---|---|
| Units | Does my logic handle every shape correctly? | No | Every save, every commit | Free, milliseconds |
| Recorded fixtures | Does my code handle real responses, including failures? | No (replays recordings) | Every commit | Free, milliseconds |
| Contract tests | Does the API still return what my fixtures say? | Yes, with a test key | Nightly, and before releases | A few credits per run |
| Smoke tests | Does production reach the API with the production config? | Yes, with the production key | After every deploy | One to three calls |
The shape matters: many units, a good number of fixture tests, a handful of contract tests, one or two smoke calls. Each layer up is slower and costs more, so it should only test what the layers below cannot. Units cannot tell you the API renamed a field; contract tests can. Contract tests cannot tell you production's key was mistyped in the deploy; a smoke test can.
Units: make the logic pure
The most valuable tests in an integration never touch the network, because most of the bugs are not in the HTTP call. They are in what you do with the response: reading nested fields, classifying failures, mapping API data into your own model. Pull that logic into pure functions (input in, output out, no I/O) and it becomes trivially testable. You already have three from this course: the key loader, verifyWebhook and redactUrl. The fourth is the one almost every integration needs.
Reading fields that may be null
APIVerve responses mark some fields premium. On a plan that does not include them, the field is present with the value null, not missing. So is any field the source simply has no value for. That has a consequence people miss: null can appear at any level of a path. If data.countries[0].name were null, then data.countries[0].name.common throws a TypeError in naive code, and optional chaining (?.) is the start of the fix but not the end, because the next line usually does value || 'Unknown' and turns a real false or 0 into the fallback.
Country Lookup shows both kinds of field in one response. cca3 is available on every plan; landlocked, latlng and independent are premium:
curl "https://api.apiverve.com/v1/countrylookup?country=France" \
-H "x-api-key: YOUR_API_KEY"landlocked is false here, a real value. On a plan without premium fields it would be null. Your code must tell those apart.
landlocked: false is an answer: France has a coastline. landlocked: null is the absence of an answer. Code that writes if (!country.landlocked) reports France as coastal on every plan and reports every country as coastal on a plan without the field. Here is a reader that walks a dotted path, treats null and undefined at any step as "no value", and returns real falsy values untouched:
// readField(body, 'data.countries.0.cca3', 'n/a')
// Returns the fallback when any step is null, undefined, missing or not an object.
// Returns real values unchanged, including 0, false and ''.
export function readField(body, path, fallback = null) {
let cur = body;
for (const step of String(path).split('.')) {
if (cur === null || cur === undefined || typeof cur !== 'object') return fallback;
if (!Object.prototype.hasOwnProperty.call(cur, step)) return fallback;
cur = cur[step];
}
return cur === null || cur === undefined ? fallback : cur;
}Array indexes work as path steps because '0' is an own property of an array. The own-property check means a path like data.constructor returns the fallback rather than a built-in function. And the final line is the whole point: only null and undefined become the fallback.
The tests write themselves, and they are the ones worth having, because they pin down exactly the cases that break in production:
// node --test
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { readField } from './read-field.mjs';
const body = {
status: 'ok',
data: { countries: [{ cca3: 'FRA', landlocked: false, ccn3: null, capital: ['Paris'], name: null, population: 0, motto: '' }] },
};
test('walks objects and array indexes', () => assert.equal(readField(body, 'data.countries.0.cca3', 'n/a'), 'FRA'));
test('keeps false', () => assert.equal(readField(body, 'data.countries.0.landlocked', 'n/a'), false));
test('keeps 0', () => assert.equal(readField(body, 'data.countries.0.population', -1), 0));
test('keeps empty string', () => assert.equal(readField(body, 'data.countries.0.motto', 'n/a'), ''));
test('null leaf gives fallback', () => assert.equal(readField(body, 'data.countries.0.ccn3', 'n/a'), 'n/a'));
test('null in the middle gives fallback', () => assert.equal(readField(body, 'data.countries.0.name.common', 'n/a'), 'n/a'));
test('missing index gives fallback', () => assert.equal(readField(body, 'data.countries.5.cca3', 'n/a'), 'n/a'));
test('walking into a string gives fallback', () => assert.equal(readField(body, 'data.countries.0.cca3.length', 'n/a'), 'n/a'));
test('null body gives fallback', () => assert.equal(readField(null, 'data.x', 'n/a'), 'n/a'));The same discipline applies to everything downstream. When a premium field feeds a decision, decide explicitly what null means: "unknown, send to review" is usually right, and "treat as the safe value" almost never is.
Code reads const coastal = !data.countries[0].landlocked;. The account moves to a plan without premium fields. What happens for a landlocked country such as Austria?
Recorded fixtures: real responses, replayed
Units test logic against shapes you constructed. The next layer tests your full call path against shapes the API actually produced. Make a real call once, save the response, and replay it in tests by injecting a fake fetch. The Associate agent course recorded data objects for an eval; for an integration you want more, because your code branches on more than data:
- the HTTP status and the whole envelope, since the client reads
statusanderror; - the headers your code reads: the credit headers and
cf-ray, and nothing else; - metadata: when it was recorded and with what parameters, so a reader can tell how stale it is.
// node record.mjs (needs APIVERVE_TEST_KEY; run by hand, commit the output)
import { writeFileSync } from 'node:fs';
const KEEP_HEADERS = ['x-api-remaining-credits', 'x-api-credits-used', 'x-api-max-credits', 'cf-ray'];
const CASES = [
{ name: 'country-ok', api: 'countrylookup', params: { country: 'France' } },
{ name: 'country-not-found', api: 'countrylookup', params: { country: 'Atlantis' } },
{ name: 'country-missing-param', api: 'countrylookup', params: {} },
];
const out = {};
for (const c of CASES) {
const url = new URL(`https://api.apiverve.com/v1/${c.api}`);
for (const [k, v] of Object.entries(c.params)) url.searchParams.set(k, String(v));
const res = await fetch(url, { headers: { 'x-api-key': process.env.APIVERVE_TEST_KEY } });
out[c.name] = {
request: { api: c.api, params: c.params }, // never the key
status: res.status,
headers: Object.fromEntries(KEEP_HEADERS.map((h) => [h, res.headers.get(h)])),
body: await res.json(),
recordedAt: new Date().toISOString(),
};
}
writeFileSync('fixtures/countrylookup.json', JSON.stringify(out, null, 2));Record failures as well as successes. The country-not-found case captures a real 404 with "error": "country not found", and country-missing-param captures a real 400 whose error is an object, { "missing": ["country"] }, which is exactly the case a string-assuming error handler gets wrong. Some failures you cannot or should not trigger on demand: an out-of-credits 429, a 500, a timeout. Write those fixtures by hand, modelled on the documented shapes, and label them "synthetic": true so nobody mistakes them for recordings.
Replaying is a matter of making fetch an argument instead of a global:
export function fixtureFetch(fixture) {
return async () => new Response(JSON.stringify(fixture.body), {
status: fixture.status,
headers: Object.fromEntries(Object.entries(fixture.headers).filter(([, v]) => v !== null)),
});
}
// In a test: const client = makeClient({ fetch: fixtureFetch(fixtures['country-not-found']) });Before committing any fixture, scrub it. Response bodies from validation and lookup APIs often contain the personal data you sent: an email address, a phone number, an IP. Use obviously fake inputs when recording (example.com addresses, documentation IP ranges such as 203.0.113.0/24), and run the same redaction you use for logs over the file. A fixture directory is a repository, and lesson 1 covered what happens to anything in a repository.
Contract tests against the live API
Fixtures have a weakness that grows with time: they are a photograph. If the API changes after you recorded, your fixture tests keep passing against the old shape. A contract test closes that gap by calling the live API on a schedule and checking that the response still has the shape your fixtures, and therefore your code, assume.
The key matters. Contract tests get their own sub-key from your inventory, not a developer's key and never production's:
- Blocked down to exactly the APIs under test, so a bug in the test harness cannot wander into expensive endpoints.
- Rate-capped on Mega, so a loop in CI cannot eat the window your production traffic uses. (It has its own bucket regardless, because each key does.)
- Stored only in CI secrets for protected branches and scheduled runs.
Assert shape and invariants, not volatile values. A contract test that checks a price, a timestamp or a news headline fails every day for reasons that are not bugs. Pick inputs whose answers are stable (France's ISO code, a fixed HTTP status code) and assert those exactly; for everything else, assert types and presence.
Two APIVerve-specific invariants are worth a test each:
Echo the input. Unknown parameters are silently dropped, not rejected. If you rename a parameter in your code (country to countryName) and the API does not recognise it, there is no 400 for an unknown parameter. With a required parameter gone, you get a 400 naming what is missing; with an optional one, the call succeeds and quietly ignores what you asked for. Where an API echoes its input, assert the echo: Country Lookup returns data.search as the uppercased search term, so a contract test that asserts search === 'FRANCE' proves the parameter was received.
Allow null wherever null is legitimate. A shape check that demands landlocked be a boolean fails on a plan without premium fields. Treat null as compatible with any type; what you are guarding against is a field disappearing or changing type.
// Describe a JSON value's structure, ignoring values.
export function shapeOf(v) {
if (v === null) return 'null';
if (Array.isArray(v)) return v.length ? [shapeOf(v[0])] : [];
if (typeof v === 'object') return Object.fromEntries(Object.entries(v).map(([k, x]) => [k, shapeOf(x)]));
return typeof v;
}
// List ways `actual` breaks what `expected` (from a fixture) promised. New fields are fine.
export function shapeDrift(expected, actual, path = '') {
if (expected === 'null' || actual === 'null') return []; // nullable either way
if (Array.isArray(expected)) {
if (!Array.isArray(actual)) return [`${path || '(root)'}: expected array, got ${JSON.stringify(actual)}`];
return expected.length && actual.length ? shapeDrift(expected[0], actual[0], `${path}[0]`) : [];
}
if (typeof expected === 'object') {
if (typeof actual !== 'object' || Array.isArray(actual)) return [`${path}: expected object`];
return Object.keys(expected).flatMap((k) =>
k in actual ? shapeDrift(expected[k], actual[k], `${path}.${k}`) : [`${path}.${k}: missing`]);
}
return expected === actual ? [] : [`${path}: expected ${expected}, got ${actual}`];
}// node --test contract.test.mjs (nightly; needs APIVERVE_TEST_KEY)
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { shapeOf, shapeDrift } from './shape.mjs';
import { readField } from './read-field.mjs';
const KEY = process.env.APIVERVE_TEST_KEY;
const fixtures = JSON.parse(readFileSync('fixtures/countrylookup.json', 'utf8'));
async function live(api, params) {
const url = new URL(`https://api.apiverve.com/v1/${api}`);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, String(v));
const res = await fetch(url, { headers: { 'x-api-key': KEY }, signal: AbortSignal.timeout(15_000) });
return { status: res.status, body: await res.json() };
}
test('countrylookup still matches the recorded contract', { skip: !KEY && 'no APIVERVE_TEST_KEY' }, async () => {
const { status, body } = await live('countrylookup', { country: 'France' });
assert.equal(status, 200);
assert.equal(readField(body, 'data.search'), 'FRANCE'); // the parameter arrived
assert.equal(readField(body, 'data.countries.0.cca3'), 'FRA'); // a stable fact
const drift = shapeDrift(shapeOf(fixtures['country-ok'].body), shapeOf(body));
assert.deepEqual(drift, [], `fixture drift:\n${drift.join('\n')}`);
});When the drift check fails, read the message before touching anything. A missing field or a changed type is a real break: fix the code, then re-record. A field that became null does not fail the check by design. A new field never fails it, because additive changes are not breaking. And skip uses a message, so a run without the key says so in the output instead of silently passing; in the nightly job, make a missing key an error rather than a skip.
Smoke tests and a bounded test budget
The top of the pyramid runs in production, after each deploy, with the production configuration. Its only job is to prove the deployed service can reach the API with the key it actually has: the secret is mounted, the key is not the staging one, the egress IP is on the allow-list. One cheap, deterministic call is enough:
curl "https://api.apiverve.com/v1/httpstatuslookup?code=418" \
-H "x-api-key: YOUR_API_KEY"1 credit, a fixed answer, no personal data: a good smoke test. Assert data.name and the 200.
Assert the status, one stable field, and that x-api-remaining-credits is present and above a floor. If it fails, the deploy pipeline should stop and roll back, and the log line from lesson 3 tells you which outcome class it was: auth means the key, forbidden means scoping or the IP allow-list, network means egress.
Finally, cap what the live layers can spend. Live tests are the place an innocent loop gets expensive: a parametrised test over 500 inputs, a retry wrapper inside a retry wrapper, a watch mode re-running on every save. Put a meter in front of the live client that knows each API's price and refuses to cross a per-run budget. Credits differ per API (Country Lookup is 2, HTTP Status Lookup is 1, Email Validator is 20), and each API's cost is on its marketplace page and in its definition.
const COST = new Map([['countrylookup', 2], ['httpstatuslookup', 1]]); // from each API's listing
export function testBudget(maxCredits) {
let spent = 0;
return function charge(api) {
const cost = COST.get(api);
if (cost === undefined) throw new Error(`No known cost for ${api}; add it before testing it live`);
if (spent + cost > maxCredits) throw new Error(`Test budget exceeded: ${spent}+${cost} > ${maxCredits}`);
spent += cost;
return spent;
};
}Then decide when each layer runs, so that the expensive ones run least:
on:
pull_request:
push: { branches: [main] }
schedule: [{ cron: '17 3 * * *' }] # nightly contract run
jobs:
unit-and-fixtures: # every PR, no secrets needed
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 20 }
- run: node --test read-field.test.mjs verify-webhook.test.mjs fixtures.test.mjs
contract: # nightly and on main only
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
env:
APIVERVE_TEST_KEY: ${{ secrets.APIVERVE_TEST_KEY }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 20 }
- run: test -n "$APIVERVE_TEST_KEY" # a missing key fails here instead of skipping
- run: node --test contract.test.mjsKeeping the live job off pull_request does two things. It keeps per-PR runs free and fast, and it keeps the test key away from code nobody has reviewed yet: a pull request can change the test code, and a test that can read a secret can print it. GitHub does not pass repository secrets to workflows triggered from forks for the same reason.
With a scoped, rate-capped key, a budget meter and a nightly schedule, the arithmetic is small and predictable: a dozen contract calls at 1 to 2 credits each is roughly 20 credits a night, about 600 a month, known in advance.
A contract test for Country Lookup passes every night. A refactor renames the query parameter from country to name in the client. What catches this, and how?
Course wrap-up
Across this course you have built the operating layer of a production integration. Keys are inventoried, scoped, injected from a secrets manager, rotated by overlap and revoked in seconds. Inbound webhooks are verified over raw bytes, inside a replay window, in constant time, before anything parses them. Every call leaves a structured, redacted record with an outcome class, credits and the cf-ray support needs, and alerts fire on the classes that need a person, including a credit runway that catches loops and leaks. And the test suite proves it in layers: pure units for the logic, recorded fixtures for real shapes, budgeted contract tests for drift, and a smoke call after every deploy.
That completes the APIVerve Certified Engineer path. When you are ready, take the Production Integrations exam.