The end of tokenmaxxing – and what comes next

The rise and fall of tokenmaxxing has been dramatic, not least because of how quickly it happened. From a darling of CEOs wanting to show how AI-forward they are to a balance sheet nightmare, the journey this particular strategy for measuring AI lasted mere months before its star started to fade.

In this episode of the ITPro Podcast, Jane and Bobby are joined by Frank Böhmer, CEO of Ninox, to talk about what prompted the trend in tokenmaxxing, why it fell apart, and what comes next.

Highlights

"[Gamification] leads to a pretty high stress factor for the employees, because they feel really pressed, even if it's not a direct performance metric, if it's just ... something that creates visibility, and so they feel really pressured to utilize LLMs to the highest possible amount."

"If you put this kind of as a success metric and monitor it, then it becomes useless, because people kind of optimize for that ... so even just looking at the output, so the number of tokens does not say a lot, and it's very easy to use models in a way that is not efficient. It can easily, for example, pollute the context window by just cramping in all the information you have."

Latest Videos From

Watch full video here:

"When you have to work with within the constraints of a budget or a token budget, then there are clearly ways to optimize the outcome I can create with models, for example, by providing the right information within the context of the model, by making sure that all the data that the company processes is readily available for models to work with by giving the right guidelines for teams to utilize that by giving them access to the right models in the right context."

Footnotes and sources

TOPICS

Jane McCallion is Managing Editor of ITPro and ChannelPro, specializing in data centers, enterprise IT infrastructure, and cybersecurity. Before becoming Managing Editor, she held the role of Deputy Editor and, prior to that, Features Editor, managing a pool of freelance and internal writers, while continuing to specialize in enterprise IT infrastructure, and business strategy.

Prior to joining ITPro, Jane was a freelance business journalist writing as both Jane McCallion and Jane Bordenave for titles such as European CEO, World Finance, and Business Excellence Magazine.