This digest covers LLM evaluation and observability developments from 2026-11-05 through 2026-11-12. The headline thread this week: several independent papers converge on the same uncomfortable finding — the instruments we use to measure LLM quality (human raters, LLM-judges, contamination tests, failure taxonomies) are themselves unstable, and that instability is often bigger than the signal teams are trying to measure.
🔥 Highlights
AI Isn't Getting Dumber, We're Getting Pickier: Rising Standards in Human Evaluation of Large Language Models — human judgment drifts even when the model doesn't.
MotionGraphicsBench: Evaluating Motion Graphics Generation with Programmatic Verification — deterministic checks beat a VLM-judge on accuracy.
Langfuse launches AI Gateway — one proxy swap turns every model call into a trace.
Understand how agents use your MCP server — traces the silent bugs where a tool call "succeeds" with wrong data.
Gemini Decisions API — a third judge-as-a-service API lands, and its outputs…