Book: LLM Observability Pocket Guide: Picking the Right Tracing & Evals Tools for Your Team Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub You ship a classifier. It picks one of four labels for every support ticket. It looks fine in evals at 92% accuracy. The dashboard goes green and you go to lunch. Two weeks later somebody points out that the wrong-label tickets are silently routing themselves to the wrong queue and a small fraction of refund requests are sitting in a billing inbox nobody reads. The model never said "I'm not sure." It just said refund with the same flat voice it used for the 91 other refunds that morning. The thing is, the model knew. The probability it assigned to refund on that token was 0.34. The next token, billing , was 0.31. It was a coin flip dressed up as a decision.…