Tokenminning Issue #3

Welcome to the latest edition of Tokenminning - resources to make inference optimization more effective. The newsletter is curated by Rob May at Neurometric. If you have tips or articles or case studies send them to [email protected] and we will include them here.

The best post of this past week was Ricky Ho’s long twitter post about the next chapter of AI. If you want to build for where AI is going next, his analysis is very good. Here he describes one of the core reasons the world is embracing tokenminning:

_{Perhaps the simplest way to understand this transition is through an analogy that repeatedly comes to mind whenever I think about enterprise AI adoption. When a company needs someone to reconcile its books, prepare financial statements, or process invoices, it does not recruit a PhD in pure mathematics, not because that individual lacks the ability to perform the work, but precisely because they possess far more capability than the task economically justifies. The value created by accurate bookkeeping is fundamentally capped. Once the accounts are correct, there is little incremental benefit from employing substantially more intelligence, regardless of how exceptional that intelligence may be. Rational organizations therefore optimize around economics rather than capability, hiring the least expensive person capable of consistently producing work that meets the required quality standard while deploying scarce intellectual talent toward problems where additional intelligence genuinely creates incremental value.}

Tokenminning: How To Get More From Your Chatbot For Less - Link

How to Use Local Coding Agents - Link

Are AI Employees More Expensive Than Humans? - Link

Interesting linkedin post from Hugging Face’s CEO on local models - Link

How NVIDIA’s Inference Software Stack Powers Lower Token Cost - Link

Research Paper: How Much Is Left? LLMs’ Linearly Encode Their Remaining Output Length - Link

Research Paper: ToolFailBench: Diagnosing. Tool Use Failures in LLM Agents - Link (tool call failures can dramatically drive up token usage)

Ed Zitron writes that The AI Industry is Losing. Is it? Or is the lack of focus on tokenminning just hindering ROI and all of this will right size in the next 18 months?

Thanks for reading this week and don’t forget to check out Tokenminning on Reddit.