Menu

Laravel RAG Regression Testing in CI
πŸ“°

Laravel RAG Regression Testing in CI

DEV Community: laravelΒ·Muhammad Ikram UL MustafaΒ·about 6 hours ago
#aS43vBs2
#devtolaravel#devto#dev#eval#corpus
Long article (3397 words) β€” no AI summary yet.
Reading 0:00
15s threshold

A broken queue job throws. A bad migration stops the deploy. A RAG feature that gets worse does neither: it keeps answering confidently and keeps citing sources. They are just the wrong sources, and you find out when a user does.

I built laravel-rag , an open-source Composer package, so that kind of regression fails a build instead. Its rag:eval command runs golden questions against your corpus, scores recall and MRR, and with --ci exits non-zero when a score drops below a floor you set. Version 0.3.0 rebuilt the package on the official Laravel AI SDK, and this tutorial is written for it. The laravel-rag case study covers why it exists; this post goes from install to a red GitHub Actions check.

Why retrieval regressions never throw an error

Three ordinary changes cause most of them: a different chunk size, a cheaper embedding model, or a few hundred new documents crowding the old ones out of the top results.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.