Deployment has become a poor measure of progress.
Across industries, the discussion about AI agents has shifted from whether to deploy them to how quickly more can be added.
As part of this change, a critical assumption has arisen and must now be re-examined; Managing agents and deriving value from them are not the same thing.
Chief Technology Officer (CTO) at KTSL.
Recent research found that 88% of UK businesses are actively deploying AI agents, but only 20% have achieved measurable business impact.
This is a sequencing problem rather than a technological problem.
A Bad Business Case
When AI agents first appeared on company roadmaps, the business plan was almost always built around cost reduction: automate this, reduce headcount here, reduce expenses there. But this playbook was borrowed from all previous waves of business technology, and for early pilots it was a useful framework.
Since then, organizations that have moved beyond pilot projects into real-world operations have largely abandoned it. The outcomes they care about now are faster resolution of operational issues and a better experience for the people served by these systems. Cost reduction, when it occurs, tends to be a byproduct rather than a goal.
A deployment designed to reduce costs will be measured based on costs. If the same deployment actually improved resolution speed or reduced the number of outages requested from support teams, that value would not be recorded and would not be leveraged to justify additional investment. The lesson is as old as time, but we must keep reminding it: if you have the wrong objective from the start, you can easily make a successful deployment look like a failure.
Why deployments are underperforming
A significant proportion of AI agent implementations are not meeting expectations, and a significant portion of organizations have responded by suspending further investments. Before taking this as evidence that the technology isn’t working, it’s worth examining the actual cause of this underperformance.
The most common barriers we see are lack of skills, poor business case definition, data quality issues, and lack of a competent technology partner. Again, none of this has to do with technical issues, but rather preparation and execution.
In practice, I see another problem in that agents need to be seen as genuinely better than the process they are replacing with the people doing the work. If engineers and operators don’t feel the benefits, you’ll never see effective adoption. After that, deployments will disappear before they have a chance to prove themselves. Adherence, as always, requires the same attention as technical implementation.
Defining What Success Really Looks Like
One of the consequences of deploying agents without agreed upon measures of success is the inability to demonstrate value even when it is created. This is a particular problem in IT management, where AI agents are increasingly involved in detecting, triaging and resolving incidents.
Mean time to resolution (MTTR) is the most important metric in this context, and it deserves further examination. The stages of the incident life cycle are as follows:
Identification,
triage,
isolation,
diagnosis,
repair, and
Verification
Each of these has a different weight depending on where the current process is slowest. An organization that takes ten minutes to identify an incident but two minutes to resolve it once identified has a different problem than one where diagnosis is more of a constraint. Agents must therefore be applied at the stage where they will bring a real gain in efficiency.
Establish a baseline before selecting the intervention and know where time is actually being wasted. Then you can set a specific goal to reduce it and measure that. Without this, it is truly difficult to distinguish a successful deployment from a busy deployment.
The governance deficit
Security and governance frameworks are still mostly designed for environments where humans make consequential decisions, even if software executes them. When you introduce autonomous agents, capable of accessing and acting on sensitive data in real time with limited human oversight, these frameworks become ineffective. This is not a criticism of how they were designed, but rather a description of a gap that grew as the deployment expanded.
When I look at the organizations most at risk, they tend to be those whose existing frameworks are too deeply ingrained to be easily revisited. Legacy architecture is the constraint, and larger organizations bear more of it.
There is a comparison to be made here with the era of SaaS sprawl and shadow IT. In both cases, the technology evolved faster than the controls around it, and the cost of establishing those controls retrospectively was higher than what their integration would have been. With this in mind, it is easy to understand that governance does not act as a brake on the deployment of new technologies, but rather as a prerequisite that guarantees long-term effectiveness.
Late integration decisions are costly
Enterprise IT infrastructure is so heterogeneous that technology planning tends to be underestimated. The combination of public cloud, private hosting, and hybrid environments – layered on top of existing systems running processes that are poorly documented and harder to change than anyone would like – creates conditions that require deliberate architectural thinking from the start. Agents designed without this environment in mind will require significant redesign once they encounter it.
There is also a less obvious use of AI in this process. Applied earlier in the planning cycle, it helps identify areas where legacy systems create the most friction and where investment in integration will yield the most return. Most organizations deploy AI to drive results; Fewer use it to improve the quality of decisions that shape deployments in the first place, making this agent application a competitive differentiator.
The question of sequencing
Fundamentally, the technology used in successful and unsuccessful AI agent deployments is the same. What differentiates them is the sequencing: the conditions for success have been established before the agents are put into service.
These conditions require more discipline than sophistication, including restricted use cases, clean and well-governed data, integration as a priority, and security frameworks that account for the presence of autonomous systems.
The question worth asking is whether your organization knows precisely what each AI agent is supposed to improve on, whether they improve it, and what will happen to that agent eighteen months from now if they don’t. Most businesses can’t answer these three questions. If you can, you’ll already be ahead of the game.
We list the best IT automation software.
This article was produced as part of TechRadar Pro Insightsour channel that showcases the best and brightest minds in technology today.
The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you would like to contribute, find out more here: https://www.techradar.com/pro/perspectives-how-to-submit
