Menu

1 / 3

Your Question-Level Split Can Leak the Same Answer Twice

DEV Community: nlp·vexaloom-labs·about 14 hours ago
#bSxJVMHL
#devtonlp#devto#dev#answer#question
Reading 0:00
15s threshold

Different question URLs can carry the same answer straight across your evaluation boundary.

You split threads, rerun retrieval evaluation, and get scores comfortable enough to make you uncomfortable. The partition code checks question URLs. The answer evidence has other travel arrangements.

Lab note 009: can we preserve question context without letting the same answer cross that boundary? The catch comes early: the grouping below protects relationships visible in the collected rows. It can't prove that missing provenance, paraphrases, or uncollected copies are absent.

Why can a question-level split leak answers?

Quora sometimes displays answers written for merged or equivalent questions.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.