Welcome to the latest edition of Tokenminning - resources to make inference optimization more effective. The newsletter is curated by Rob May at Neurometric. If you have tips or articles or case studies send them to [email protected] and we will include them here.
Post of the Week
The best post of this past week was this post from the Playcode blog about the real cost of frontier models:
A "token" is not a fixed amount of text. Each vendor's tokenizer cuts the same file into a different number of pieces, and you pay per piece. So $/Mtok is not comparable across vendors.
Anthropic's newest tokenizer (Sonnet 5, Opus 4.8, Fable 5) produces ~30% more tokens from the same code than their previous one. The list price did not change.
On identical files, it produces 1.36-1.73x GPT's token count. TypeScript is the worst case at 1.73x.
In effective terms, Opus 4.8's $5 / $25 behaves like $7.50 / $37.50, and Sonnet 5 after its intro window like $4.50 / $22.50.
This measures input tokenization only. Output verbosity, thinking, and caching move whole-task bills further; details near the end.
Articles To Review This Week
OpenAI and Anthropic Face New AI Reality As Users Shift From “Tokenmaxxing” To Efficiency - Link
VLLM Launches a Semantic Router - Link
What is Token Efficiency: A Practical Guide For AI Teams - Link
Research On Lossless Compression - Link
Satya Nadella on the Reverse Information Paradox - Link
Key Metrics For Evaluating Token Efficiency in AI Systems - Link
The Harness Effect: How Orchestration Design Sets The Token Economics of Enterprise Agentic AI - Link
A Guide To Token Engineering
I wrote a post on the Neurometric blog about “What Does A Token Engineering Platform Do?”
Token engineering is the practice of treating tokens as an engineered resource: measured, benchmarked, routed, and continuously optimized. It’s a systems discipline, not a procurement exercise.
The common misconception is that token engineering means “use a cheaper model.” It doesn’t. It means optimizing every AI workload across three dimensions simultaneously: cost, speed, and reliability. Sometimes the right answer is a smaller, cheaper model. Sometimes it’s a faster one. Sometimes it’s the frontier model, but with a compressed prompt and an aggressive caching layer in front of it. The point is that the answer is different for every task, and it changes constantly.
Have an article that should be included? Send it to [email protected]