Over the past month building Penned —a free AI humanizer and detector designed to rewrite AI drafts into natural prose while strictly preserving facts, names, and numbers—we ran into a fascinating, counter-intuitive reality when evaluating our candidate picker against GPTZero v3 API ( version 2026-10-07-base ) .
Most people (and even many ML practitioners) imagine AI text detectors as sentence-level statistical classifiers: each sentence has a perplexity or burstiness score, and the document score is simply an average or a cumulative tally.
Our empirical benchmarks on research paper drafts revealed something completely different: GPTZero's verdicts are document-level, non-additive, and heavily order-dependent.
Here are 7 empirical findings from our evaluation logs ( eval/gptzero/FINDINGS.md ), and what they mean for anyone building or evaluating NLP systems today.
1.…