There's a new approach that:

cuts corpus size by 40x.

reduces tokens per query by 3x.

improves vector search relevance by 2.3x

And it doesn't touch your retrieval algorithm, your reranker, or your embedding model. It fixes something upstream that almost no one examines

Every RAG pipeline starts with the same assumption: a chunk of text is the right unit of knowledge to embed

That assumption is almost never examined

And it's the source of most of the retrieval failures people try to fix downstream

Why the Chunk Is a Bad Unit

A chunk of text is a structurally neutral container. It knows nothing about:

where its ideas begin or end

which version of a document it came from

who is allowed to see it

Since a chunk has no idea boundary, the splitter cuts wherever the token count runs out.

You end up retrieving half a table, or a conclusion with no argument, or a claim stripped of the context that makes it true.…