Menu

1 / 2

Cutting LLM Calls in Incident Triage: Sliding Windows, Log Reduction, and a pgvector Cache

DEV Community: java·Vedas Kudalkar·14 days ago
#dvujdJAO
#devtojava#devto#dev#errors#model
Reading 0:00
15s threshold

A Kafka-to-WebSocket pipeline where the model is the last resort, not the first step.

The problem

The obvious way to add an LLM to log monitoring is to send logs to a model and ask what is wrong. That breaks quickly. Most log lines are not errors. During an outage, the same error repeats thousands of times with only timestamps, UUIDs and counters changing. A raw stack trace with framework frames is mostly noise. And the same database timeout will show up on orders today and inventory tomorrow, so you pay for the same diagnosis again.

I built telemetry-incident-dashboard around one rule: every step before the model must be cheaper than the model, and must be able to stop the pipeline.

It is a working prototype.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.