Menu

What 586 Real Traces Taught Us About LLM Agent Determinism
πŸ“°

What 586 Real Traces Taught Us About LLM Agent Determinism

DEV Community: machinelearningΒ·Vishwajitsinh MahidaΒ·12 days ago
#q92MnodK
#devtomachinelearning#education#devto#dev#temp
Reading 0:00
15s threshold

What 586 Real Traces Taught Us About LLM Agent Determinism

How temperature selectively degrades agent reasoning β€” and what that means for observability.

The Setup

On November 3, 2026, we ran 586 traces of a LangGraph folder-research agent against AgentPulse, our metadata-only observability platform. The agent used Ollama with qwen2.5:7b on an RTX 4060 (Legion), plus 6 traces from a Pixel 8 Pro running qwen3b via llama.cpp in Termux.

The experiment: same 10 questions, same corpus, four temperatures (0, 0.3, 0.7, 1.0). We measured trace shapes β€” span counts, durations, tool calls β€” to understand how sampling temperature affects agent behavior.

Why this matters: If you're building observability for AI agents, you need to know what "normal" looks like. Without a determinism baseline, every alert is a guess.

Finding 1: Temperature 0 Is Deterministic

At temp=0, qwen2.5:7b produced identical trace shapes across 20 runs per question. Nine out of ten questions had standard deviation 0.0 on span counts.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.