Safety Rules Don’t Survive the Context Window: Quantifying and Fixing Agent Compliance Decay
Injected Safety Rules Are Ignored as Context Windows Fill
Anyone deploying LLM agents with hardcoded safety rules has seen this: as dialogs grow, initial guardrails disappear. The cause is structural. Once the system prompt—“Never give medical advice,” “Decline illegal requests,” etc.—is pushed out of the model's context window, it vanishes from generation. Agents get riskier, and compliance disintegrates, regardless of careful prompt engineering.
The problem appears in both controlled experiments and production logs. For example: in a 2023 production deployment using GPT-4 via OpenAI API, a medical chatbot with explicit “do not diagnose” system instructions started outputting symptom assessments once the dialog exceeded 8,000 tokens. The rules weren’t absent in config—they were just out of context.…