Your local model gives you an answer that looks reasonable.
But do you trust it?
When you’re reading the response yourself, you can judge that on the spot. But what if the response feeds straight into downstream components and nobody looks at it?
It’d be great if we could give that answer a second look, before putting it to work.
In fact, a decision model can do just that.
Decision models are close relatives of LLMs. The difference is that LLMs are typically used to generate text, while decision models are specialized in selecting from defined choices given the context and the question.
Here, we can give the decision model the local model’s answer and the original context, then ask what should happen next: should we keep the answer? Or request a revision? Or flag it for human review?…