The Cutout Quality System
How Esy makes clip art that is truly transparent — the render pipeline, the math that removes a background's color and not just its pixels, the quality gates that judge images the way a human does, and the eight experiment waves that set every threshold. Written so an engineer who has never seen this system can take it over.
What this system makes, and why it is hard
A clip-art asset has one non-negotiable property: it must work on any background — a mug, a t-shirt, a dark website. That means the background of the generated image must be removed completely: no leftover patches, no holes punched through the artwork, and no colored fringe around the edges.
The hard part is the last one. Deleting background pixels is easy; the background's color also hides inside the image — mixed into soft edges, showing through translucent things (bubbles, glass, watercolor washes), and blended into dark outlines. Delete the pixels and skip the color, and every image ships with a faint stain of whatever it was rendered on. That stain, in our case a purple one, is what most of this system exists to prevent.
The pipeline
Esy renders every subject on a solid key color — a synthetic green screen, exactly like film studios use. Because we write the prompt, we control the background; and because we control the background, removal becomes arithmetic instead of guessing. This matters more than it sounds: the foundational computer-graphics result (Smith & Blinn, 1996) proves that separating a subject from an unknown background is mathematically unsolvable in general — while a known backing color makes it tractable. Most background-removal products guess with a neural network. Ours only falls back to guessing when the arithmetic refuses.
The two lanes
| Lane | How it removes | When it runs |
|---|---|---|
| Keyed (chroma-key) | Deletes exactly the key color, by measured distance in linear light. Deterministic; enclosed gaps (wreaths, frames) come out transparent by construction. | Default for solid-fill styles. The planner picks the key per item, guaranteed absent from the palette. |
| ML (segmentation) | A neural network (BiRefNet family) predicts what is subject and what is background. | Line/sketch styles where paint-time key contamination is worse than segmentation errors — and every rescue when the keyed lane refuses. |
Despill: removing the color, not just the pixels
After cutting, the keyer runs despill — it subtracts the key's color cast from every visible pixel of the foreground. This is not our invention and not optional: it is the second half of the original 1971 chroma-key patent (Vlahos, US 3,595,987), and every professional keyer — Nuke, Ultimatte, After Effects — performs it automatically. We initially only cleaned edge pixels; production taught us why the textbooks say "the whole foreground": dark strokes blended with the key count as opaque and dodge edge-only cleanup, and translucent content carries the key through itself.
The audit: gates that see what you see
Every cutout passes a measurement gate before it can ship. The core lesson from the academic matting literature (and from our own bruises): naive pixel-count metrics anticorrelate with human judgment — a batch can score "pass" while any person calls it stained. So the flagship metric judges what the viewer judges: it composites every visible pixel over a dark background exactly like the gallery does, converts to a perceptual color space (CIE Lab), and counts pixels a human would call key-colored.
| Metric | Plain meaning | Fails above |
|---|---|---|
visibleKeySpillRatio | Of the pixels a viewer actually sees, what fraction is perceptibly key-colored? | 0.02 |
subjectKeyTintRatio | Key-hued fraction of fully-opaque subject pixels (dark ink + key blends). | 0.08 |
keptPocketAreaRatio | ALL leftover background blobs inside the subject, summed. (A single-blob cap exists too — one image once passed with 28 individually-small pockets covering 60% of it.) | 0.05 total / 0.01 largest |
largestDamagedHoleAreaRatio | Biggest hole punched through real artwork — judged against the raw render, so a wreath's see-through center is never damage. | 0.005 |
edgeHaloRatio | Near-white (or near-matte) fraction of the 2px edge band — the classic white halo. | 0.25 |
Every threshold cites its calibration: the failing population's scores, the clean population's scores, and the margin between them. A threshold nobody can trace is a threshold nobody can safely tune.
The file format lesson
The strangest bug in this system's history: images that were clean in memory shipped stained. The cause was the save step. Lossy WebP inherits a rule from its video-codec ancestor: it stores brightness for every pixel but color only once per 2×2 pixel block — like a coloring book that allows one crayon per four squares. If three squares are blue fox and one is the leftover key hiding in an invisible pixel, all four get a purple-ish crayon. No quality setting turns this off; it is welded into the format.
The rules that fell out, now enforced at the single shared encoder: anything with transparency is stored as lossless WebP (pixel-exact, no color averaging, still ~26–42% smaller than PNG per Google's own study), with edge colors flooded outward into the transparent region first ("alpha bleeding," the same fix game engines ship). Opaque images keep the smaller lossy encode — they have no edges for color to smear across.
The prompt rule: never name the forbidden thing
Image models do not process negation. When key color bled into ink shading, we tried telling the model the subject must contain no magenta — and it painted more magenta: outlines, glows, a shadow puddle. Forbidding the devices ("no outline, no glow") made it worse still; contamination tripled across the two "hardened" prompts. The rule, now pinned by a test: the render directive names the background color exactly once, neutrally — and quality problems are solved by routing, math, and gates, never by prompt prohibitions.
Eight waves: what each one taught
| Wave | What we saw | What it turned out to be | The rule it left behind |
|---|---|---|---|
| 1 | Every keyed run died instantly. | An edge-polish step re-corrected already-clean keyed edges until its own safety check refused them. | Corrective steps must check what corrections already happened (mechanism provenance). |
| 2 | Perfect audits, but six runs died at a safety check. | Compression noise on hot-pink edges misread as corruption. | Calibrate safety checks on the physics of the failure they exist to catch. |
| 3 | Scoreboard perfect; human eyes caught a green rim and a magenta-tinted bicycle. | Contamination lived in pixels no metric watched. | When eyes beat the metrics, build the metric. |
| 4–5 | Prompts hardened against contamination; batches got worse each time. | Negative prompts amplify — naming the forbidden color paints more of it. | Never name the forbidden thing; state the wanted state once. |
| 6 | Style-routing decisions looked settled. | Half the evidence came from the poisoned waves 4–5. | Check what pipeline version produced a measurement before trusting it. |
| 7 | Purple rims everywhere — while every gate said pass. | Three stacked causes: edge-only despill, report-mode gates, and the lossy encoder staining files at save time. | Despill the whole foreground; enforce by default; lossless storage for transparency. |
| 8 | Verification probe of the full new stack. | 10/11 runs at exactly 0.0000 visible spill on stored files; the ladder rescued a bad render unattended. | The system as documented on this page. |
Operating the system
Benching. A 30-prompt golden set — engineered to stress white fills, enclosed shapes, wispy edges, thin lines, and key-colliding palettes — lives in the API repo as a reviewable script. Runs cost about a cent each; a 12-prompt probe answers directional questions, the full set settles routing decisions.
Records. Every experiment wave files an audit record (run ids, metrics, verdicts) to the organization's shared context, so the next engineer — or the next agent — starts from conclusions instead of re-deriving them. Behavior-changing findings also get a lab note telling the story.
Per-batch QA reports. Every pack order files an automatic report on completion: what the gates caught (by failure class, with run ids), what the ladder rescued, retries spent, and the batch's worst reading per metric against its ceiling. The gates' work is visible per-batch — a quality system you can't see is a quality system you can't trust.
When to re-bench. Any render-model change, any keyer/encoder/metric change, any routing decision, threshold-adjacent drift in a batch report — and, above all, whenever a human disagrees with a "pass." Eyes outrank metrics; the correct response to the disagreement is a new metric, not an argument.
Where the source of truth lives
The pipeline and its tests live in the api.esy.com repository — the cited engineering rules in docs/CUTOUT_STANDARDS.md, the record-keeping convention in docs/CLIPART_QUALITY_AUDITS.md, and the bench driver under scripts/bench/. The evaluation standard, the wave audit records, and the lab notes live as knowledge artifacts in the ESY LLC organization context, where every operator and agent inherits them. This page is the narrative map; those are the law.