Sunil Ganesh
Head of Experience-Led Transformation
6 min read
Your team can now take a product brief and produce five plausible interface directions before lunch. Working ones, with states, copy and edge cases handled.
That isn’t the problem. The problem is that they can produce all five without anyone establishing that any of them deserves to exist, and the artifacts are now good enough to hide the absence.
Which is the real shift. The most dangerous property of AI-generated design work is that it looks finished. Fluency was always our proxy for effort, and effort was our proxy for thought. That proxy is gone.
Meanwhile the cost of building fell far enough that the old gate went with it. When a prototype took a sprint, someone had to justify it. The cost was the decision point. Remove the cost and you remove the decision point.
The scarce thing in AI product development is no longer making. It’s grounds, and supplying grounds is exactly what your experience team is for.
What your scorecard doesn’t cover
Ask what your experience team delivered last quarter and you’ll get a list. Every item on it is real work worth measuring. But each is either an activity count or a downstream proxy, and none of them looks at the decision itself.
| What you already measure | What it tells you |
|---|---|
| Artifacts delivered | Throughput |
| Research studies completed | Rigor of inquiry |
| Design system adoption | Consistency of execution |
| Usability benchmarks | Build quality, months after the decision that mattered |
| Stakeholder satisfaction | Working relationship |
| Conviction Score | Whether the work changed what got built, and how quickly |
A team can score well on every row and still not have moved a single thing that was going to be built anyway. That’s the gap this UX metric closes, and the only one it claims to.
The instrument
The Conviction Score applies to one product or service initiative before build commitment. Five questions, scored 1 to 5.
Intent. Is the job articulated independently of any solution? A feature request is not a problem statement.
Observed behavior. Has a real person used a working version? When prototypes are nearly free, “we couldn’t build it to test it” stopped being an answer.
Failure. Do you know what happens when the system is wrong, what the person does next, and where the human-in-the-loop seam sits? Failure here isn’t an edge case, it’s a distribution. The recovery path is the service.
Trust. Do you know what this person needs to see to believe an output they can’t check? “They’ll get used to it” is not an answer to an AI trust problem.
Consequence. If it’s wrong, what happens to the person, not the metric? Most teams apply the same evidence bar to a recommendation carousel and to something that touches someone’s money.
You are scoring the strength of your evidence on each dimension, not the attractiveness of the answer. Below fifteen out of twenty-five, we’d treat an initiative as not ready. That threshold is a starting heuristic rather than a validated cutoff, and worth recalibrating after twenty initiatives. The value was never in the number, but in forcing teams to expose where conviction is missing.
What it looks like
An insurer wants an AI claims assistant: it reads submitted documents, assesses the claim against policy terms, then settles, requests more information, or routes to a human. Strong business case. Convincing prototype.
| Dimension | Score | Basis |
|---|---|---|
| Intent | 4 | Settle simple claims without the wait. Well evidenced from complaint data. |
| Observed behavior | 2 | Six users shown a demo. Nobody submitted a real claim through a working version. |
| Failure | 1 | Escalation exists in the architecture, not in the experience. |
| Trust | 2 | The interface shows a decision, not what it rested on. |
| Consequence | 1 | A wrong denial affects someone's finances. Same evidence bar as the marketing site. |
| Conviction Score | 10 / 25 |
The score doesn’t say the assistant is a bad idea. Intent scored 4, so the problem is real. It says the organization is about to fund a build on two questions it hasn’t answered, and those two determine what happens to a customer when the system gets it wrong.
It also says what to do next. Not another three-week study: Intent is already the strongest dimension, so more discovery would move the score least. What’s needed is a working version in front of real claimants, a designed recovery path, and a decision about what evidence a person needs before accepting a machine’s judgment on their claim. Three targeted pieces of work, each measured in days.
Inside the experience KRA
These four indicators belong on the experience team’s own KRA, alongside the quality and throughput measures already there. This is not a separate governance process that happens to the team. It is how the team accounts for itself.
Coverage. Share of initiatives scored before build commitment. A low number usually indicts the operating model, not the team.
Lift. Score movement from first assessment to build gate. Inheriting a 9 and delivering an 18 beats being handed a 17.
Time-to-conviction. Median days to a defensible commitment or a documented no. The one your sponsors actually feel.
Reshape rate. Share of initiatives changed or stopped because of the assessment. A team that never changes anything isn’t exercising judgment; it’s providing cover.
None of these can be met by producing more work. They move only when what gets built changes.
Two things that decide whether it works
Who scores it. If the team scores its own initiatives and is measured on lift, the rational move is to score low on day one. So score in a room rather than a spreadsheet, with design, product and QE together, record the basis alongside every number, and have the product owner sign the opening score so nobody can discount it later.
What it sits beside. This is one indicator, not a replacement scorecard. It says nothing about craft, consistency or accessibility. Keep your quality and throughput measures; this one design KPI adds the axis they don’t cover. Run it on ten to twenty initiatives with no consequence attached before it goes anywhere near a performance conversation.
The question underneath
Go back to the five interface directions your team produced before lunch. Any of them could be built by Friday. That was never the constraint.
The constraint is that somebody has to be able to say why this one, and be able to say it to a board, and be right. Experience measurement has spent a decade asking did we design it well. The harder and more useful question is whether the organization became more confident about what it should build, and more willing to stop what it shouldn’t.
That is what your experience team is actually for, and until now there has been no line on the scorecard where it shows up.
Frequently asked questions
What is the Conviction Score?
A five-question assessment applied to a single product or service initiative before build commitment. It scores the strength of your evidence on intent, observed behavior, failure, trust and consequence, out of 25. Below 15, treat the initiative as not ready to fund.
How is this different from the UX metrics and design metrics we already track?
Existing design metrics tell you about throughput, consistency and build quality, and all of them are worth measuring. None of them looks at the decision itself, which is whether the work changed what got built. That is the one axis this adds.
Where does it sit relative to a customer experience measurement program?
Customer experience measurement tells you how something performed after it shipped. This runs before the commitment, so the two are sequential rather than competing. Keep both.
Is this only useful for AI initiatives?
No, but it matters most there. In AI-native product development the failure and trust dimensions carry most of the risk, because the person is being asked to accept an output they cannot verify.
Does scoring initiatives slow down AI product development?
Time-to-conviction is on the indicator list precisely because it should do the opposite. A documented no in nine days is faster than a build that gets unwound in month five.
Who should own the score?
Score it in a room with design, product and QE present, not in a spreadsheet owned by one function. Have the product owner sign the opening score so it cannot be discounted later.