Tests that call a real LLM are slow, cost money on every run, and fail randomly because the output changes. So many teams quietly skip testing their AI features altogether.
The fix is to notice that most of an "AI pipeline" isn't the model at all. Building the prompt, retrieving context, parsing the reply and deciding what to do next are ordinary, deterministic code, and ordinary code is easy to test.
1. Put the model behind an interface
Make your pipeline depend on a small interface instead of a specific SDK:
from typing import Protocol
class LLM ( Protocol ) : async def complete ( self , prompt : str ) -> str : . . .
class FakeLLM : def __init__ ( self , reply : str ) -> None : self . reply = reply self . prompts : list [ str ] = [ ]
async def complete ( self , prompt : str ) -> str : self . prompts . append ( prompt ) return self . reply
Enter fullscreen mode
Exit fullscreen mode
Production passes in the real client.…