The KPI That Never Changed When the Agent Arrived

A support team hit its quarterly target for the first time all year: cases resolved per person, up. Nobody asked how many of those cases the agent drafted and how many the person actually solved. The bonus went out anyway.

Carlos Andrés Ramírez · 2026-09-16

It happens in support, in credit origination, in contract review: any team where someone measures how many units of work a person produces in a week. An agent shows up, takes on part of the volume, and the number leadership watches keeps climbing. Nobody stops to ask where the new number comes from, because asking means admitting the old KPI no longer measures what it claims to measure.

The easiest move is to leave it alone. The KPI already exists, already has an approved target, already sits on the dashboard leadership checks every month. Touching it means reopening an uncomfortable conversation about how much of the result is the person's doing and how much belongs to the license the company already pays for. It is simpler to let the number climb on its own and hand out credit as if nothing had changed.

And meanwhile the KPI keeps rewarding volume without saying where it came from, quietly punishing whoever still reviews every hard case carefully, and missing the fact that the person has stopped practicing the very judgment that metric was supposed to be cultivating.

The symptom

How do you redesign a team's KPIs when an agent does part of the work?

It shows up in the leadership meeting before it shows up on the dashboard. The number climbs quarter after quarter and the explanation is always the same: "the agent is helping." Nobody measures how much. Last year's target, set when the team was fully human, stays identical or climbs by decree, without anyone recalculating how much of that volume can actually depend on a person's judgment.

The number went up and nobody knows how much of it the agent produced versus the person.
This quarter's target is last year's target with a different decimal point, and nobody explains where the adjustment came from.
A case the agent marked closed comes back weeks later as a complaint, and it never gets subtracted from anything.
The person no longer touches the simple cases that used to teach them the job, and nobody tracks whether their judgment on the hard ones is improving or stalling.
The bonus formula is the same one from last year, when a person produced the entire volume.

The problem underneath

The number went up. What it measured didn't.

A volume KPI never measured volume for its own sake. It measured, indirectly, a person's effort and judgment: how many cases someone who actually knows the job could handle well in a normal week. That hidden relationship is exactly what breaks the moment part of that volume stops coming from a person and starts coming from a system that doesn't get tired, doesn't learn from an angry customer, and never risks a performance review over a single case.

And the issue isn't that the number is rising. It's that it keeps rising even while the human judgment behind it stalls or erodes, and nobody can tell the two apart by looking at the dashboard alone.

A KPI that can't tell you who produced the result isn't measuring the work. It's measuring that someone, or something, closed the case.
BECOME

The framework

What has to be redesigned in the KPI, and in what order?

Redesigning the KPI doesn't mean inventing a new metric from scratch. It means separating four pieces that currently live blended into a single number.

Split the number, don't blend it: The volume the agent resolves and the volume the person resolves get counted separately, even if they land on the same dashboard. A single number that adds both together can't tell you whether the team improved or the company simply paid for more machine.
Actually resolved, not just closed: A case counts as resolved when it doesn't come back. If the agent marks it closed and the customer reopens it two weeks later, that case gets subtracted from the original count, not added as a new one once someone finally handles it.
Recalculated target, not inherited: This quarter's target isn't last year's target with a rough adjustment. It gets recalculated from how much a person can actually produce today with the agent already in place, and it gets revisited every time the agent's scope changes.
Judgment that keeps developing: If the agent absorbs the simple cases, the person still needs somewhere to keep practicing the job. The metric has to explicitly track their performance on the hard cases that still reach them, not just the team's total volume.

None of the four requires a new measurement system. It requires whoever owns the KPI to sit down with the current number and decide, with these four questions in front of them, what to separate before setting next quarter's target.

Take the KPI of a team that already works with an agent and ask whoever runs it how much of the number the agent produced. If the answer is "I don't know," or only comes after digging through an improvised spreadsheet, that's the first piece that needs splitting.

Frequently asked questions

Does a team's KPI need to change the moment an agent joins?

Not the same day, but before the next performance period's target gets set. A KPI designed for an all-human team stops measuring what it claims to measure once part of the volume comes from a system, and using it unchanged hands out credit and blame based on information that's no longer true.

How do you separate what the agent produces from what the person produces inside the same number?

By tagging each case at the source, not estimating it afterward. The system that routes the work already knows whether the agent solved it, the person solved it, or the agent started it and the person finished it; that data needs to travel all the way to the dashboard instead of getting lost in the total count.

What does it mean for a case to be "actually resolved" instead of just closed?

That it doesn't come back. A case closed within the response window that the customer reopens, escalates, or repeats the following week wasn't resolved, it was filed away. The metric needs to subtract those cases from the original count instead of adding them separately once someone finally deals with them.

How do you measure whether a person's judgment keeps developing once the agent absorbs the simple cases?

By isolating their performance on the hard cases that still reach them and tracking that figure over time instead of the team's total volume. If that rate holds or improves, the judgment is still developing. If it stalls or worsens, the person has stopped practicing the very skill the original KPI assumed it was building.

Let's redesign your team's KPI

From the idea to the operation

Redesigning the process before automating it is direction and operating design work, not a tooling decision.

About the author

Carlos Andrés Ramírez — Transformation Director

Specialist in business transformation and reinvention. Director of Specialised Programmes and lecturer in Artificial Intelligence at UPC's Graduate School.