Tessa Vandermeer

I keep this section to the unglamorous tools that sit in front of everything else, and I hold a position about them. A count of words, a count of tokens or a formatter that reshapes a prompt should not need an account, a subscription or a network round trip to do its job. Token counting deserves the most attention because it is where the arithmetic stops being obvious. A tokenizer splits text into units, and the split depends on the model family. The same paragraph produces different counts for different models, sometimes by a wide margin, and the byte-to-token ratio is worse for code and structured data than for prose. Any number you quote needs the tokenizer named alongside it. Counting locally also lets you answer the question a dashboard cannot: whether this prompt will fit before you send it. Context windows are large and not infinite, and the arithmetic that matters is what remains after your system prompt, tool definitions, retrieved documents and the conversation history have all taken their share. Formatting tools earn their place by making prompts readable and diffable. A prompt you can read is a prompt you can review in a pull request, and a change to a prompt is a change worth reviewing like any other code. What I will not build is anything that silently truncates. A token counter that reports a ceiling and says nothing about it is worse than no counter. Every tool here either handles the full input or tells you precisely where it stopped.