Frontiers of compute: how to cut AI inference costs

By McKinsey & Company.

This email, sent by Bill Wiseman and Marc de Jong, global leaders of McKinsey's Semiconductors practice, announces a new article titled "Frontiers of compute: The technologies to reduce AI inference costs".

The central message the email previews is brief but forceful: AI's next big breakthrough may not be a smarter model, but a cheaper token. In other words, the article focuses on the technologies that make inference cheaper (the process of running an already-trained model to generate responses), as opposed to the usual emphasis on training ever-larger models.

The body of the email does not elaborate on which specific technologies the full article covers; it simply presents the headline, the hook, and links to the main piece along with two suggested related articles under "Also Consider": "The next era of semiconductor value creation" and "Where AI will create value—and where it won't".

Broadly speaking, as sector context, reducing inference costs is a topic of growing strategic relevance for the semiconductor and agentic AI industries, since the mass deployment of agents and LLM-based applications depends largely on the cost per inference token continuing to fall, which directly affects the economic viability of scaling these systems in production.