The end of tokenmaxxing – and what comes next

The rise and fall of tokenmaxxing has been dramatic, not least because of how quickly it happened. From a darling of CEOs wanting to show how AI-forward they are to a balance sheet nightmare, the journey this particular strategy for measuring AI lasted mere months before its star started to fade.

In this episode of the ITPro Podcast, Jane and Bobby are joined by Frank Böhmer, CEO of Ninox, to talk about what prompted the trend in tokenmaxxing, why it fell apart, and what comes next.

Highlights

"[Gamification] leads to a pretty high stress factor for the employees, because they feel really pressed, even if it's not a direct performance metric, if it's just ... something that creates visibility, and so they feel really pressured to utilize LLMs to the highest possible amount."

"If you put this kind of as a success metric and monitor it, then it becomes useless, because people kind of optimize for that ... so even just looking at the output, so the number of tokens does not say a lot, and it's very easy to use models in a way that is not efficient. It can easily, for example, pollute the context window by just cramping in all the information you have."

"When you have to work with within the constraints of a budget or a token budget, then there are clearly ways to optimize the outcome I can create with models, for example, by providing the right information within the context of the model, by making sure that all the data that the company processes is readily available for models to work with by giving the right guidelines for teams to utilize that by giving them access to the right models in the right context."

Footnotes and sources

• Could rising token costs boost interest in on-premises hardware?
• Uber’s eye-watering AI bill shows enterprises are ‘still measuring AI success through consumption rather than outcomes’ – and it's warping our perception of ROI and productivity
• IT leaders are being stung by "unexpected" AI costs
• The AI pricing time bomb
• ‘What we’re seeing right now is just rapid escalation in AI token spend’: Accenture tells staff to stop using AI for unnecessary tasks amid surging costs
• Surging AI costs could exceed developer salaries by 2028 – analysts say context engineering could be the key to optimizing token consumption
• Companies are regretting investing in generative AI so quickly – here’s how to avoid buyer’s remorse
• Goodhart's law - Wikipedia