Menu

Case Study: A Cassette Proxy That Records a Model Once and Replays It in CI
πŸ“°

Case Study: A Cassette Proxy That Records a Model Once and Replays It in CI

DEV Community: pythonΒ·Dakota LiuΒ·11 days ago
#AxEtvoUu
#devtopython#devto#dev#record#replay
Reading 0:00
15s threshold

Your integration tests should not need a network, an API key, or a budget to prove that your prompt-building code still assembles the right payload. The cheapest way to get there is to call the model exactly once, store the response under a hash of the request, and replay that response forever after. This is a case study of that small tool: what it does, the code that implements it, where it breaks, and who should walk away from it.

The project is deliberately tiny β€” one file, standard library only, three modes β€” because a record/replay layer that you cannot read in ten minutes is a layer you will not debug at 2 a.m. Treat the timings below as things you measure on your own hardware, not as published benchmarks.

Background: the test suite that quietly became an HTTP client

A prompt-assembly module usually starts life as pure functions: you build messages, you template a system prompt, you trim history to a token budget.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.