Use one trace context to connect a failed request, its tenant cohort, and the decision to roll an experiment back. Keep the context immutable across the request, attach it to structured error events, and keep cohort labels out of high-cardinality metrics. That combination gives an on-call engineer a useful path from an error to a rollout decision without turning observability data into a second patient-data system.
I build email, SMS, and OTP flows, so I am suspicious of any design that treats delivery failure as a single log line. A timeout may be a dependency problem, a rate-limit response, or a cohort-specific configuration error. The same caution applies to a healthtech experiment: the trace is evidence for a rollback, not proof that every failure has one cause.
The trade-off is real.
What must remain true during a rollback?
The architecture has three invariants. Every inbound request gets a valid correlation context or an explicitly generated one.…