Short answer: capture one structured error event per failed request, but retain a compact, redacted index longer than the diagnostic payload. In a nightly healthtech pipeline, the bill is usually driven by event count multiplied by retained bytes, not by the act of detecting an exception. Reduce noise at the source: classify failures, fingerprint them deterministically, attach correlation IDs, and keep PHI out of the event. Then make resolution a replayable, auditable workflow rather than a search through unbounded text.
What is the bill actually made of?
Start with the retention equation:
retained cost ≈ error events × average event bytes × retention duration
That is the whole accounting problem.
That equation is deliberately plain. A pipeline that logs every retry, stack trace, request body, and upstream response can create a large searchable archive even when the number of distinct defects is small.…