AI ‘overhyped’ for productivity gains: Gartner

The time for viewing artificial intelligence (AI) as a simple means to solve productivity woes is over, according to Gartner vice president analyst Luke Ellery, who believes the technology needs to be placed instead on specific business cases.

Speaking to ARN, Ellery said he views the claims of productivity gains through AI to be overhyped in the industry, as it’s a difficult concept to measure.

“If you’ve got 10,000 staff and you give everyone an AI tool, how are you actually going to harvest that?” he said.

“What we do see is that there’s an uptick in employee satisfaction typically, if you release a group AI tool, but you’re not really going to get productivity.”

As a result, Ellery said the focus should be on targeted opportunities.

“Instead of just throwing AI at your employees, maybe you’ve got a specific problem that’s data rich and resource intensive that you can actually focus and test out a business case,” he said.

“It might be a bit hard, but organisations should be trying to, when they’re selling services to provide the information and the insights, help build out those business cases in areas that are outside productivity.”

In fact, Ellery claims, there aren’t enough examples of AI being implemented that explicitly show productivity gains. He also said the rise of DeepSeek earlier this year, a considerably more affordable solution with a smaller footprint compared to models from other companies, has contributed to skepticism towards the technology in general.

As such, Ellery expects more of a focus to be placed on traditional business use cases”, rather than the “venture capitalist approach” previously seen in the market.

“We’re at a point now this year, especially [in the current] economic cycle with a lot of geopolitics and a lot more uncertainty around, that now is the time if you’ve got an AI solution or AI capabilities to really demonstrate what those benefits are, but not just looking at productivity,” he added.

Drifting past hallucination

When integrating AI models into these specific business use cases, Ellery also said to be wary of the technology’s innate downfalls. While this typically includes hallucinations – the generation of inaccurate information – drift is a less obvious, but still very real, concern.

“Drift occurs over time and it sometimes occurs because we try and help the AI model meet our expectations by creating synthetic data,” he said.

“We might find a model has got, as an example, a bias against women, so we put some synthetic data in there to try and balance it out so it’s more even, but then maybe the AI overemphasises that synthetic data and all of a sudden it’s got a bias against men.

“That’s where we see drift and that’s where there’s more operational responsibilities. It’s like having a whole team of staff and you want to monitor them. You don’t just let them go and not manage them — you invest in the management of staff.”