Most structured-extraction tutorials look the same. Take a document, write one big prompt that says "extract A, B, C, D, E, F", get JSON back. Done.

This works on short inputs.

It quietly breaks on long ones.

After running this in production for a while, I stopped doing it. Here's what I switched to and why.

The fat prompt problem

Say you have a 50-page report and you want a structured summary out of it. The natural first move is something like:

Extract: - title - sections (with headings) - purpose - mentioned services - acceptance criteria - ... Return JSON in this shape: { ... }

Enter fullscreen mode

Exit fullscreen mode

You hand the whole document to the model. It returns JSON. It looks fine on the first try.

Then you scale it up and three things happen:

Quality drifts. The model "forgets" mid-document. Later sections are summarized worse than earlier ones, or fields go missing.

One bad field poisons the whole call.…