An integration that works is invisible. The first time anyone looks closely is when it stops working, and at that point the questions are always the same: which calls failed, since when, with what status, how slow were they before they failed, how much did they spend, and can the provider see the same request on their side? If your logs cannot answer those in a minute, you are debugging by guesswork during an outage.
This lesson builds the instrumentation that answers them: one structured log record per API call, redaction so those records are safe to keep, metrics and SLOs derived from the records, alerts that fire on the failures that need a human, and the identifiers that connect your log line to APIVerve's.
One structured record per call
Free-text logs like API call failed: 429 are readable by a person and useless to a query. A structured log is one JSON object per event with consistent field names, so "error rate per API over the last hour" is a filter and a count rather than a regular expression.
For an API call, the record worth writing contains:
| Field | Source | Why |
|---|---|---|
api | Your call site | Group and alert per endpoint |
method, url (redacted) | Your call site | Reproduce the call |
status | HTTP status | The coarse outcome |
outcome | Your classifier | The class you alert on (below) |
latency_ms | Your clock, around the call | SLOs and timeouts |
credits_remaining, credits_used, credits_max | x-api-remaining-credits, x-api-credits-used, x-api-max-credits | Burn rate and runway |
cf_ray | cf-ray header | The edge request id; present on every response |
trace | x-cloud-trace-context header | The origin request id, when the call reached the API |
attempt | Your retry loop | Separate first tries from retries |
error | The envelope's error | What went wrong, in the API's words |
Two details from the live API shape that code. First, x-api-credits-used is the credits spent so far in the billing cycle, not the cost of this call; x-api-remaining-credits is what is left, and x-api-max-credits is the cycle's allowance. Second, the envelope's error is not always a string. A missing required parameter comes back as a 400 whose error is an object:
{ "status": "error", "error": { "missing": ["country"] }, "data": null }
A logger that does error.toLowerCase() or concatenates it into a message throws or prints [object Object] at exactly the moment you need it. Serialise whatever is there.
The outcome class is the field that makes alerting possible. Collapse statuses into the handful of situations that need different responses:
ok:2xxwithstatus: "ok".bad_input:400, or a404. On lookup APIs a404usually means nothing was found for that input; a404on every call means the API id itself is wrong. Either way, your request, not an outage.auth:401. The credential is missing, unrecognised, deleted, expired or suspended.forbidden:403. A key restriction refused the call (blocked API, IP allow-list).rate_limited:429with credits remaining.out_of_credits:429withx-api-remaining-creditsequal to0. Check fornullfirst, becauseNumber(null)is0.server:5xx.network: no response at all (DNS, reset, timeout).
Here is a wrapper that writes the record. It keeps your existing call logic and adds only observation:
// Node 18+, ESM.
import { redactUrl } from './redact.mjs';
const BASE = 'https://api.apiverve.com/v1';
function classify(status, body, remaining) {
if (status === 0) return 'network';
if (status >= 200 && status < 300 && body?.status === 'ok') return 'ok';
if (status === 400 || status === 404) return 'bad_input';
if (status === 401) return 'auth';
if (status === 403) return 'forbidden';
if (status === 429) return remaining !== null && Number(remaining) === 0 ? 'out_of_credits' : 'rate_limited';
if (status >= 500) return 'server';
return 'other';
}
const num = (h) => (h === null ? null : Number(h));
const asText = (e) => (typeof e === 'string' ? e : JSON.stringify(e)); // error can be an object
export async function observedCall(api, { method = 'GET', params = {}, key, attempt = 1, timeoutMs = 10_000, log = console.log } = {}) {
const url = new URL(`${BASE}/${api}`);
const init = { method, headers: { 'x-api-key': key }, signal: AbortSignal.timeout(timeoutMs) };
if (method === 'GET') {
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, String(v));
} else {
init.headers['content-type'] = 'application/json';
init.body = JSON.stringify(params);
}
const started = performance.now();
let res = null;
let body = null;
let networkError = null;
try {
res = await fetch(url, init);
body = await res.json().catch(() => null);
} catch (err) {
networkError = err.name === 'TimeoutError' ? 'timeout' : err.message;
}
const h = (name) => res?.headers.get(name) ?? null;
const status = res?.status ?? 0;
const record = {
event: 'api_call',
api,
method,
url: redactUrl(url.toString()),
status,
outcome: classify(status, body, h('x-api-remaining-credits')),
latency_ms: Math.round(performance.now() - started),
credits_remaining: num(h('x-api-remaining-credits')),
credits_used: num(h('x-api-credits-used')),
credits_max: num(h('x-api-max-credits')),
cf_ray: h('cf-ray'),
trace: h('x-cloud-trace-context'),
attempt,
error: networkError ?? (body?.status === 'ok' ? null : asText(body?.error ?? `HTTP ${status}`)),
};
log(JSON.stringify(record));
return { record, body };
}Notice what is not in the record: the request headers (which hold the key), the request body (which may hold personal data), and the response data (large, and often personal). Log the shape of the call, not its contents. When you need contents to debug a specific case, sample a small, redacted fraction into a separate, short-retention store.
A dashboard groups failures by the status field alone. Overnight, 429s jump from 20 to 4,000. What can you conclude without more data?
Redact before you write
Logs are copied everywhere: aggregators, alert emails, support tickets, laptops of whoever was on call. Anything in a log has a much wider audience than the system that wrote it. Two kinds of data must never land there unmasked: secrets and personal data.
Secrets are the easier rule because there is no legitimate reason to log them at all. The APIVerve key travels in a header, and the wrapper above never logs request headers. But URLs are where other secrets hide: OAuth codes, signed-link signatures, session tokens, password-reset tokens, and keys for services that do accept them in a query string. A URL is logged by default by almost every HTTP library, proxy and framework, so redact URLs before they reach any logger.
const SENSITIVE = /(key|token|secret|password|passwd|pwd|auth|session|signature|sig|code|credential)/i;
const MASK = 'REDACTED';
// Masks sensitive query values (and any user:password) in a URL string.
// Leaves everything else byte-for-byte as it was, and never throws.
export function redactUrl(url) {
if (typeof url !== 'string' || url === '') return url;
let s = url.replace(/^([a-z][a-z0-9+.-]*:\/\/)([^/@?#]*)@/i, (_, scheme) => `${scheme}${MASK}@`);
const hashAt = s.indexOf('#');
const fragment = hashAt === -1 ? '' : s.slice(hashAt);
if (hashAt !== -1) s = s.slice(0, hashAt);
const q = s.indexOf('?');
if (q === -1) return s + fragment;
const query = s
.slice(q + 1)
.split('&')
.map((pair) => {
const eq = pair.indexOf('=');
const rawName = eq === -1 ? pair : pair.slice(0, eq);
let name = rawName;
try { name = decodeURIComponent(rawName.replace(/\+/g, ' ')); } catch {}
return eq !== -1 && SENSITIVE.test(name) ? `${rawName}=${MASK}` : pair;
})
.join('&');
return `${s.slice(0, q + 1)}${query}${fragment}`;
}
// redactUrl('https://x.example/cb?code=abc123&state=ok&api_key=apv_9f3c')
// -> 'https://x.example/cb?code=REDACTED&state=ok&api_key=REDACTED'The choices are deliberate. It works on the string rather than round-tripping through URLSearchParams, which would re-encode every other parameter and make the logged URL differ from the one you sent. It decodes the parameter name before matching, so api%5Fkey does not slip past. It matches names by substring, so x-api-key, access_token and client_secret are all caught, and it errs toward masking: a false positive costs a little debugging detail, a false negative is a leaked credential. It masks user:password@ in the authority. And it never throws, because a logger that throws on a malformed URL hides the very error you were trying to record.
Personal data is harder because it is the payload. An email in a validation call, an IP in a lookup, a name in a sanctions check. Three layers keep it out of logs:
- Do not log payloads by default. The wrapper logs the redacted URL, and GET parameters are in the URL. For APIs whose parameters are personal (email, phone, IP), log a hash or a truncated form of the parameter instead, or add those parameter names to the sensitive list.
- Mask free text with local rules on the hot path. A few regular expressions for emails, phone numbers and card numbers, run on every message you log, cost microseconds.
- Use a detector where rules fall short. For free text you store or forward (support tickets, form submissions, error messages that quote user input), APIVerve's Data Masking API finds and replaces several kinds of personal data in one call:
curl -X POST "https://api.apiverve.com/v1/datamasking" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"text":"signup failed for jane.roe@example.com from 203.0.113.7, callback 555-123-4567"}'data.masked replaces each item with a placeholder; data.detected counts what was found by type.
Data Masking costs 10 credits per call, so it belongs on text you keep, not on every log line. A log pipeline that calls a paid API per line has turned your logging volume into a bill.
Metrics and SLOs
Metrics are what you get by counting the records. From the fields above you can derive, per API:
- Request rate, split by
attemptso retries do not hide inside traffic growth. - Error rate by outcome class. Not one "error rate": a
bad_inputspike means your data changed, anauthspike means a key changed, aserverspike means the dependency is struggling. - Latency percentiles: p50, p95, p99. Averages hide the tail, and the tail is what trips your timeouts. Different APIs have very different normal latencies (a lookup and a PDF render are not comparable), so keep percentiles per API.
- Credit burn: the drop in
credits_remainingper hour.
A service level objective turns those into a promise you can check. Pick a service level indicator that reflects what your users experience, a target, and a window. For a dependency, a useful SLI is "the share of calls whose outcome is ok, bad_input or forbidden", because those are the calls where the dependency did its job; your own bad input should not count against the provider, and a scoping refusal is your configuration. A target like 99.5% over 30 days gives an error budget of 0.5%: about 3.6 hours of total failure a month, or a steady trickle of failures adding up to the same. When the budget is burning faster than the window allows, that is an alert. When it is healthy, the occasional 500 is not worth waking anyone for.
Keep the dependency SLO separate from your own service's. If an upstream API misses its target, your service can still meet yours through caching, fallbacks and graceful degradation, and those are exactly the controls the SLO tells you to invest in.
Alerts that need a human
An alert should mean "a person must act now". Everything else is a dashboard. These are the alerts worth having for an API integration, each tied to an outcome class:
| Alert | Condition | Usual cause | First action |
|---|---|---|---|
| Credential broken | Any auth outcome in production for 2+ minutes | Rotation missed a system, expiry, deleted sub-key | Check which key (fingerprint), check the error sentence |
| Scope refused | forbidden rate above a small threshold | New egress IP, a new API not on the key | Read the message: it names the API or the IP check |
| Out of credits | Any out_of_credits | The month is spent | Stop retries, alert the owner, upgrade or wait |
| Dependency degraded | server + network share above budget burn rate | Upstream incident | Collect cf_ray values, check status, degrade gracefully |
| Credit runway | Projected exhaustion before renewal | A loop, a leak, or real growth | Find the API and key driving burn |
The credit runway alert is the one people skip, and it is the one that catches both a runaway retry loop and a leaked key before the month is gone. Compute it from two samples of credits_remaining and the renewal time. The x-api-renewal response header carries it as Unix seconds when the account has a renewal date on record; if it is absent, use your billing date.
// Two samples of x-api-remaining-credits at least an hour apart, plus x-api-renewal (Unix seconds).
export function creditRunway(earlier, later, renewalSec) {
const hours = (later.at - earlier.at) / 3_600_000;
if (hours <= 0) return null;
const burnPerHour = (earlier.remaining - later.remaining) / hours;
const hoursToRenewal = (renewalSec * 1000 - later.at) / 3_600_000;
if (burnPerHour <= 0) return { burnPerHour: 0, hoursLeft: Infinity, alert: false };
const hoursLeft = later.remaining / burnPerHour;
return { burnPerHour, hoursLeft, alert: hoursLeft < hoursToRenewal };
}The dashboard's 80% usage warning is a backstop, and you should leave it on. It fires once per cycle at a fixed point; the runway alert fires on the day the burn rate changes, which is the day something went wrong.
Production calls start returning 401 at 09:12, three minutes after a teammate rotated the primary key. Sub-key traffic from the nightly job is unaffected. What is the most likely cause?
Correlating a failure with APIVerve support
When a failure is on the provider's side, the fastest path to an answer is giving them the exact request. APIVerve responses carry two identifiers:
cf-rayidentifies the request at the edge. It is present on every response, including ones the edge answered itself (a401for a missing key never reaches the origin, and still has acf-ray).x-cloud-trace-contextidentifies the request at the origin. It is present when the call reached the API, and absent when the edge answered.
There is no x-request-id. You may see that name in an access-control-expose-headers list, but the header itself is not set, so code that keys on it reads null every time. If your platform expects a request id field, populate it from cf-ray.
What to send support, which your log record already holds: the cf-ray (and the trace value if present), the UTC timestamp, the API id, the HTTP status and the error body. Optionally the key fingerprint from lesson 1, or simply whether it was a sub-key. Never the key itself, and never a screenshot of a terminal that shows it. A key pasted into a ticket is a leaked key, and you would be starting lesson 1's incident sequence.
A pattern that pays for itself: when your own service returns an error caused by a dependency, include the upstream cf_ray in your error response or your own trace. Your users quote your request id to you; your logs map it to theirs; you quote cf-ray to the provider. Three hops, no guessing.
What you have now
One structured record per call with an outcome class, credits, latency and both request identifiers; URLs and payloads redacted before they reach a logger; per-API metrics and a dependency SLO with an error budget; alerts that map each failure class to a first action, including a credit runway that catches loops and leaks early; and a support hand-off that never includes a secret. The last piece is proving all of this works before production does it for you: Test API integrations against real APIs.