Menu

1 / 3

I Audited My AI Agents and Found That Most of Their Reasoning Wasn’t Observable

DEV Community·Nic Lydon·6 months ago
#RqVPG8sx
#ai#observability#agents#langfuse#executor
Reading 0:00
15s threshold

I run a personal AI platform with eight active agents, dozens of processors, and a fully self-hosted Langfuse instance. I built the observability layer myself. I shipped it a few weeks ago. Last week I ran the audit query for the first time.

The agents that talk to me the most only had Langfuse-level lineage coverage for about 13% of their decisions.

This is the writeup of what I found, why it happened, and the schema and code that explain it. If you run agents and you've never run this audit, you have a very good chance of finding the same gap.

The Setup

Quick context. The platform is called Nexus. It's a TypeScript monorepo plus a fleet of Python processors, running on a couple of mini PCs in my apartment. It ingests 26 data sources, runs 8 reasoning agents on schedules, and serves an MCP tool surface I use as my daily driver.

Two layers matter for this post:

The agents are reasoning entities. They read from gold-layer tables, decide things, and write proposals to inbox tables.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.