Menu

JSON Schema Field Order: My vLLM Classifier Answered Before Thinking
📰

JSON Schema Field Order: My vLLM Classifier Answered Before Thinking

DEV Community: python·jidonglab·1 day ago
#mypcWvpQ
#devtopython#devto#dev#reasoning#schema
Reading 0:00
15s threshold

My ticket classifier had a reasoning field, and the reasoning was good. It would write "the customer was charged twice for the same invoice, this is a billing problem" and then file the ticket under bug .

Not once. Dozens of times.

The Pydantic model was correct. reasoning came first, category second. I checked it three times. The bug was a cache helper that sorted my JSON schema keys alphabetically, and in the alphabet, category comes before reasoning . That one sort flipped the JSON schema field order , and the model started committing to a label before it had written a single word of thought.

TL;DR

With constrained decoding (vLLM guided_json / response_format: json_schema , Outlines, xgrammar, llama.cpp grammars), JSON schema field order is generation order . The model writes the keys in the order the schema lists them.

LLMs generate left to right. If category comes before reasoning , the label is fixed before any reasoning tokens exist.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.