Table of Contents

Manual Tracing, Scores, and Evaluation with Langfuse (Self-Hosted)

Why Manual Tracing Matters for LLM Observability

Decorator vs Manual Tracing: When to Use Which

Manual Tracing with the Langfuse Low-Level API

Why Manual Tracing Matters (Even If You Use Decorators)

Full Manual Tracing Implementation with Langfuse

Code Walkthrough: Langfuse Manual Tracing Pipeline

Creating Manual Traces in Langfuse

Running the Langfuse Manual Tracing Script

Viewing Manual Traces in the Langfuse Dashboard

Manual vs Decorator Tracing in Langfuse

LLM Evaluation Metrics and Quality Scoring with Langfuse

Adding LLM Evaluation Metrics Beyond Manual Tracing

Code Walkthrough: evaluation_metrics.py

Running the LLM Evaluation Metrics Pipeline

Conceptual Mockup: Evaluation Trace in Langfuse

Real Trace from Our Self-Hosted Langfuse Dashboard

Why LLM Evaluation Metrics Matter

vLLM Diagnostics and Health Checks for LLM Observability

What the vLLM Health Check Script Validates

Code Walkthrough:…