Screen, describe and clean up user-uploaded images

The day you add an image upload to your product, you become a publisher of whatever people send. Most of it will be fine: avatars, product photos, screenshots. Some of it will not, and the first explicit image that appears on a public profile page will be screenshotted and shared long after you remove it.

This lesson builds a screening step for uploads with four APIs, each answering a different question about an image:

None of them is a judge. They are classifiers that are usually right and sometimes confidently wrong. The design question running through every section is the same: what happens to the user, and to you, when the model is wrong?

How these APIs take images

Input styles differ across the four, and it matters for how you wire them:

• nsfwimagedetector and imagecaption take a file upload (multipart/form-data, field image), JPG, PNG or GIF.
• facedetect and backgroundremover take either a file upload in image or a public image URL, as url or in image itself. They also accept WebP, and backgroundremover takes BMP.

For screening user uploads you will usually have the bytes in memory already, so uploading is natural. The URL style is convenient for images already in your bucket or returned by another API.

Screen for explicit content first

NSFW Image Detector returns two booleans on every plan, isNSFW and isSafe, which are always opposites. On plans that include them, confidence (0 to 1, how likely the image is NSFW) and category (sfw or nsfw) let you build a middle band.

Screen first because it is cheap and because it gates everything else: an image that fails here should never reach a captioning model, a public URL or a CDN cache.

The decision is not binary in a well-run system. Use three outcomes:

• Reject clear violations, with a neutral message ("This image can't be used. Please choose another.").
• Hold for review anything uncertain: the image is stored privately, the user sees "processing", and a person decides.
• Publish the rest.

With only the booleans, "uncertain" has to come from context instead of a score: new accounts, accounts with previous rejections, or images going somewhere highly visible (a public profile, a marketplace listing) can be held for review even when isSafe is true, until the account has a track record. With confidence, set two thresholds and send the band between them to review.

False positives cost real users. Swimwear, medical images, breastfeeding photos, classical art and skin-toned products all trip explicit-content classifiers. If a false positive silently deletes a photo, the user has no idea why their upload vanished. Hold and review; never silently delete.

False negatives cost you. No classifier catches everything, so keep a "report this image" link on every user image and a takedown process behind it.

Draft alt text with captions, then let people edit it

Image Caption returns one field, caption: a sentence describing the image, such as "A golden retriever running through a grassy field on a sunny day".

The obvious use is alt text, and it is a good one, with a constraint. A generic caption describes what is visible; good alt text describes what matters in context. On a product page, "a blue ceramic mug on a wooden table" is a fine start; the alt text the customer needs is "Harbor mug in navy, 350 ml, matte glaze". So:

• Pre-fill, do not publish. Put the caption in the alt text field of the upload form, where the user or editor can correct it. Most people will improve a draft; almost nobody writes alt text from a blank field.
• Never caption before screening. A caption of an explicit image is explicit text, stored and possibly indexed.
• Treat it as user-visible content. Captions can be wrong about people's age, gender or actions. If your product shows the caption publicly, someone will eventually be described inaccurately.

Captions are also useful internally: searching a media library for "whiteboard" or "receipt", or flagging uploads whose caption suggests they are not what the form asked for (a screenshot of text where you expected a product photo).

Remove backgrounds for product and profile photos

Background Remover cuts the subject out and returns a PNG. Omit bgColor for a transparent background, or pass a hex colour such as #ffffff to fill it, which is what most marketplaces want for product listings.

1
2
3
4

curl -X POST "https://api.apiverve.com/v1/backgroundremover" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"image":"https://upload.wikimedia.org/wikipedia/commons/2/26/YellowLabradorLooking_new.jpg","bgColor":"#ffffff"}'

The response gives width and height (same as the input), background (transparent or the colour used), mimeType (always image/png), expires in milliseconds and downloadURL. Premium adds foregroundRatio, the share of the image kept as foreground, and outputSize.

At 40 credits this is the most expensive call in the course, so be deliberate:

• Run it once, when the seller uploads, and store the result. Never on page view.
• Keep the original. Cut-outs fail on fine hair, glass, and objects the same colour as the background; sellers need a way to revert.
• foregroundRatio, where available, is a cheap quality check. A value near 0 means almost everything was removed (the model lost the subject); near 1 means almost nothing was. Either is worth a human look.
• Transparent PNGs are large. If the result will sit on a white page anyway, ask for #ffffff and convert the PNG to JPEG or WebP with the converter from the first lesson.

A background job for product photos, sending the seller's uploaded image by URL and keeping both versions:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

import os
import requests

KEY = os.environ["APIVERVE_API_KEY"]


def make_cutout(source_url, out_path, bg_color="#ffffff"):
    res = requests.post(
        "https://api.apiverve.com/v1/backgroundremover",
        headers={"x-api-key": KEY},
        json={"url": source_url, "bgColor": bg_color},
        timeout=60,
    )
    body = res.json()
    if not res.ok or body.get("status") != "ok":
        raise RuntimeError(f"cutout failed ({res.status_code}): {body.get('error')}")
    data = body["data"]

    ratio = data.get("foregroundRatio")  # premium; None on plans without it
    if ratio is not None and (ratio < 0.05 or ratio > 0.95):
        return {"status": "review", "reason": f"foregroundRatio {ratio}"}

    png = requests.get(data["downloadURL"], timeout=60)
    png.raise_for_status()
    with open(out_path, "wb") as f:
        f.write(png.content)
    # Keep source_url as the original so the seller can revert.
    return {"status": "done", "path": out_path, "width": data["width"], "height": data["height"]}

Put it together: one screening step for uploads

Order the checks by cost and by consequence: cheap, blocking checks first; expensive, optional work last and only on images that will be kept.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

// Node 18+, ESM. npm install express@4 multer@1
import express from 'express';
import multer from 'multer';

const KEY = process.env.APIVERVE_API_KEY;
const upload = multer({ storage: multer.memoryStorage(), limits: { fileSize: 5 * 1024 * 1024 } });
const app = express();

async function ask(api, file, extra = {}) {
  const form = new FormData();
  form.append('image', new Blob([file.buffer], { type: file.mimetype }), file.originalname);
  for (const [k, v] of Object.entries(extra)) form.append(k, String(v));
  const res = await fetch(`https://api.apiverve.com/v1/${api}`, {
    method: 'POST',
    headers: { 'x-api-key': KEY },
    body: form,
    signal: AbortSignal.timeout(30_000),
  });
  const body = await res.json();
  if (!res.ok || body.status !== 'ok') throw new Error(`${api} ${res.status}: ${body.error}`);
  return body.data;
}

app.post('/avatar', upload.single('photo'), async (req, res) => {
  const file = req.file;
  if (!file || !/^image\/(jpeg|png|gif)$/.test(file.mimetype)) {
    return res.status(400).json({ error: 'Upload a JPG, PNG or GIF.' });
  }

  let nsfw;
  try {
    nsfw = await ask('nsfwimagedetector', file);
  } catch (err) {
    // Screening unavailable: hold, never publish unscreened.
    return res.status(202).json({ status: 'held', reason: 'screening unavailable' });
  }
  if (nsfw.isNSFW) return res.status(422).json({ status: 'rejected', error: "This image can't be used." });

  const faces = await ask('facedetect', file, { confidence: 0.6 }).catch(() => null);
  const caption = await ask('imagecaption', file).then((d) => d.caption).catch(() => '');

  // Store file.buffer privately here, then publish.
  res.json({
    status: 'accepted',
    hint: faces && !faces.hasFaces ? 'We could not find a face. Is this the photo you meant?' : null,
    altDraft: caption,
  });
});

app.listen(3000);

Look at the failure handling, because that is the part that matters in production:

• NSFW screening failing closed. If the classifier is down or times out, the image is held, not published. An upload that waits an hour is an inconvenience; an unscreened explicit image on a public page is an incident.
• Face detection and captions failing open. They are advisory. If either fails, the upload still succeeds without the hint or the alt text draft.
• Background removal is not in the request path at all. For product photos, queue it as a background job after acceptance, so the seller is not waiting on the slowest, most expensive call.

Consent and transparency

Image analysis touches people who never signed up for your product: the bystander in a photo, the person in someone else's profile picture. A few commitments keep you on the right side of both law and users' expectations.

• Say what you do. Your privacy notice should state that uploads are automatically screened for prohibited content and, if you use it, analysed for faces, and which processors handle that.
• Purpose limitation. Use each result for the decision it was made for. A face check for "is this a profile photo" does not become data for advertising or identity matching later.
• Retention. Keep classifier results as long as you need them to explain a decision (a rejection appeal, a moderation audit), then delete them.
• An appeal path. Any automated rejection should have a human route back. That is good practice everywhere, and in some jurisdictions, decisions made solely by automated processing that significantly affect people come with explicit rights.

You now have the full image toolkit: converting formats, rendering images from HTML and Markdown, reading text out of them, and screening what users send you. The final course of the certification puts documents to work in larger workflows, with QR codes, barcodes, invoices and charts: Automate documents.