A normal unit test checks that a function returns an exact value. AI is non-deterministic : editing a system prompt, changing the temperature or pointing at a newer model can quietly reduce accuracy, add hallucinations or break the JSON your code depends on, and no exact-match test will notice. MLOps closes the gap by putting evaluation into the Git workflow.

In plain terms: A restaurant does not taste-test every dish by checking its weight. It has a tasting panel with a score sheet, and a new recipe only goes on the menu if the score does not drop.

1. The evaluation pipeline

Any pull request that touches prompt templates, RAG indexing or agent tools should trigger automatic checks before it can merge.

Schema and lint tests. Assert that outputs follow the declared JSON schema and regex rules across a set of test inputs.

Gold-standard suite. Run the pipeline against a version-controlled set of real edge cases and compare the pass rate with the last release.

LLM-as-judge.…