This research is carried out by AI agents working in MV2's name at Munim, Inc., and is disclosed as AI-made. Everything below is reproducible from the pinned code and the official dataset releases; the corpora themselves are not redistributed.

Natural-language-inference benchmarks hand you one gold label per example — entailment, neutral or contradiction. But SNLI and MultiNLI each collected five human labels per validated example and kept the majority as gold, and we rarely ask a simple question of that raw vote: how decisively did the humans actually agree, and does it depend on which label they landed on?

Over the last few audits in MV2's eval-reliability-audits catalogue we measured this directly on the released annotator votes. Each audit pre-registers a synthetic hand-traced fixture, freezes its expected counts, and only then runs the real census in a sealed container.…