Among the giants of the digital world, hyperscalers like Amazon, Google, Meta and Microsoft will collectively allocate more than $500 billion in capital expenditure to AI infrastructure this year.

When a handful of players invest upwards of half a trillion dollars in data centers, primarily to support AI, that tells us they see a ton of demand.

We can say with total certainty that they have every intention of recouping their investment (and then some).

CEO, AVEVA.

Indeed, the voracious demand expected from enterprises for AI capability is what has made hyperscalers, frontier labs and GPU makers darlings of the stock market.

However, their anticipated revenues are the future cash flows of companies, including those in industrial sectors like manufacturing, energy and transportation.

The Cost Shift We’re Not Talking About Enough

Up until recently, much of the discussion around AI costs has focused on training large language models. The cost of training just one OpenAI model, GPT-4, for example, is said to have been north of $100 million.

That’s a big number. It’s also largely irrelevant to the industrial organizations that will never train a frontier model themselves.

Inference, the process of applying models in live operating environments to generate recommendations, support decisions and take actions, is a different matter, however. Yet it’s largely absent from most boardroom conversations.

Unlike training, which mostly represents a single upfront investment, inference tasks to support decisions and operational execution run continuously.

As companies shift from experimentation to large-scale deployments of AI across the value chain, inference will soar as a share of workloads and IT budgets.

Tokenomics Take Root

Most businesses do not understand the true cost of AI from a financial point of view. Industry needs a new economic calculus: “tokenomics.”

Tokens—the units through which generative AI systems process and generate information—are the standard mechanism to measure and charge for AI consumption. They’re like the kilowatt-hours for which electric utilities charge their customers.

An AI agent that performs tasks across large operational datasets consumes lots of compute (and storage, networking, electricity, water and more), and the total bill reflects that increased usage.

As companies move more and more to agentic systems, these costs will mount quickly. Agentic workflows require several model calls for each task, involve large context windows and need multiple reasoning loops. By their nature, they are many times more compute-intensive than the generative AI tools most enterprises have used so far.

A Familiar Pattern

Recent research suggests the AI inference market will more than double over the next five years.

Many of us witnessed a similar pattern earlier in our careers during the move to the cloud. In the 2010s when cloud computing was reaching scale, the same economic phenomenon was at play: unit costs dropped sharply while total usage skyrocketed.

The result was, contrary to many expectations, significantly increased total IT spending. There were many business benefits associated with cloud, but it didn’t mean that total costs declined.

The coffers of hyperscale cloud providers swelled. Those industrial enterprises that did not accurately project cloud consumption curves were caught off guard. Today, tokenomics is proceeding on a similar trajectory, only faster.

By now, CFOs understand well the economics of cloud computing and how to optimize the variable cost structures they represent. A similar level of budgetary scrutiny is appropriate for AI.

Token prices are declining as a result of intense rivalry and efficiency improvements. Open-source frameworks, custom silicon and innovative development methods are driving down model costs rapidly, especially in China.

However, as companies look to integrate AI into mainstream industrial operations, plant floors and logistics networks, the absolute number of tokens generated will far outstrip gains in per-unit costs. What looks like a bargain on a per-transaction basis today will look very different as industry marches towards greater autonomy.

A Pragmatic AI Approach

Moreover, data-rich environments like industrial plants are sensitive to latency in ways that consumer and enterprise applications are not. Continuous inference over a network of sensors, industrial equipment and control systems requires a very different kind of computation than running a chatbot. The data volumes are different. The infrastructure requirements are different. And the economics are different.

This should not be taken as an argument against the deployment of AI in industrial operations. On the contrary, with a pragmatic mindset about AI’s costs, we can harness the exciting innovations AI introduces and enable all-new use cases.

The good news, though, is that tokenization does not have to mean runaway consumption. Where models are invoked, how often they run, what context they draw on and how intelligence is distributed across edge, cloud and hybrid environments all affect cost.

Importantly, AI in industry is not monolithic. Lighter-weight models, edge-based processing and other techniques will continue to improve the efficiency of AI.

Tried-and-true AI-based predictive maintenance, computer vision and process optimization solutions all provide clear illustrations of mature, winning business cases. Generative, agentic, physical and multimodal AI, however, herald something different and more complex financially.

The upside of AI in industry is significant. So are the potential costs.

They should be accounted for on the same spreadsheet.

This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit