Case study
Zero Issues Is a Claim, Not a Result
What a compliance tool does at the edge of its own certainty matters more than how often it's right.
Haim Repael · 3 min read

STAR AT A GLANCE
- Situation
- The accessibility plugin reported zero issues on a frame with eleven unlabeled images, and invented a fourth status nobody had approved.
- Task
- Make the tool honest about what it does not know, before designers stopped trusting it.
- Action
- Made "Needs input" a real status, flipped the empty default, and added a one-click correction for misclassifications.
- Result
- About 85% right on the first pass, honest about the rest, and a number a designer can act on.
I ran a scan on a real product screen with the accessibility plugin I built for Figma. It came back clean. Zero issues under Non-text Content, the rule that says every meaningful image needs a text alternative.
For a second I almost moved on. Then I looked at the frame again. Eleven images. A hero illustration, two icons doing real work, a chart. Not one had alt text saved.
The zero wasn't a finding. It was silence, reported as safety.
SituationTwo failures in two costumes
Somewhere in an early sprint, someone, probably me, had to decide what an empty result should show before the rule had run against real content. Zero felt neutral. It isn't. It told a designer everything was fine when nothing had been checked.
A few weeks later the same instinct showed up in a different disguise. The Focus Visible check wasn't returning pass or fail. It returned "Guidance." Sixteen buttons, all Guidance, with an honest note: a static file can't verify this deterministically.
The honesty was real. The status wasn't. Our own spec allowed exactly three answers: pass, fail, not applicable. Guidance was a fourth state nobody had approved. It counted toward nothing, and a designer facing sixteen items without a verdict had no idea whether the screen needed work.
And then there was the noise. One frame returned forty-one contrast issues: a background rectangle, a decorative vector, a text layer nobody would ever read. The target-size check flagged a decorative mask as a failing touch target. Something close to half of what it reported wasn't real.
A designer doesn't run statistics on a tool's error rate. They check five results, find two wrong, and stop trusting the other thirty-nine.
TaskFix trust before accuracy
The instinct, once you see a tool being wrong, is to chase accuracy. Tighten the model, catch more cases. That's correct eventually. It is not the first problem.
Trust isn't an average. It is set by the worst thing a designer notices in the first sixty seconds. So the task was to give the tool an honest way to say "I don't know", instead of going quiet or inventing a middle ground that answers to nobody.
ActionA better vocabulary for uncertainty
- "Needs input" became a real status. Not a disguised fail and not a silent zero. Pass and fail claim knowledge. Needs input claims only that a question exists and hasn't been answered yet.
- The empty default flipped. An image with no saved alt text never shows zero again. It shows a count of items needing input, every time, until someone answers.
- A one-click correction. When the tool misclassifies something, the designer can say "you got this wrong, it's a checkbox" without leaving the scan. Every mistake becomes a data point instead of a dead end. None of the accessibility plugins I'd looked at offered that.
Both bad defaults had one thing in common: neither was ever a decision. Zero was a placeholder that outlived its sprint. Guidance was a compromise reached under deadline. Neither reached a design review, because nobody thinks of empty states and fallback text as design. They are. They're what a user meets at exactly the moment trust is decided.
ResultA worse-looking number you can act on
The number that mattered wasn't raw accuracy. It was the combination: about 85% right on the first pass, plus a correction path a designer can use mid-scan. A tool that's right most of the time and honest about the rest earns more trust over a month than one that claims full coverage while quietly getting a fifth of its answers wrong.
I still open that Non-text Content panel out of habit, half expecting the old zero. It says something else now: eleven images, none labeled, waiting on someone.
What travels
This isn't specific to accessibility. Security teams call it alert fatigue: research on security operations keeps finding false-positive rates of 50 to 80 percent across common tooling. Google has published that 16% of its tests show measurable flakiness, and that 84% of pass-to-fail transitions in its CI are not real bugs.
Different domain, same design failure:
A system that can't tell its user which findings are certain will eventually be trusted for neither.
If design leadership only shows up for the confident states, it's absent for the ones that matter most.