PDFs are the worst input format for a RAG pipeline. The text is stored as positioned glyphs, not as paragraphs, so a naive extraction gives you broken lines, columns mixed together and tables turned into word soup. Then your chunks are bad, your embeddings are bad, and the model answers from garbage.
Converting each PDF to Markdown first fixes most of this: headings give you natural chunk borders, tables stay readable, and the text is clean enough to embed.
Here are the four problems that hurt retrieval the most, how to check for each, and two ways to do the conversion.
1. Two-column pages read in the wrong order
Papers and journals use two columns. A simple extractor reads straight across the page, so line 1 of the left column is glued to line 1 of the right column.
Check: open the output of a two-column paper and read the second paragraph. If the sentences stop making sense halfway through a line, the columns are mixed.
2.…