LLM Foundry: the boring stack that makes an LLM actually useful

Most AI projects are built backwards.

People start with the model and only later discover they needed a memory system, semantic retrieval, tool use, tests, and a fallback plan for when one provider decides to nap for no visible reason.

That is the part I care about now.

LLM Foundry is the workshop around an LLM — not the model itself. It is the layer that makes a model useful for actual work instead of just looking smart in a demo.

What changed

The current version now has a few things worth showing instead of just claiming:

semantic retrieval backed by embeddings, so memory search is not just keyword matching

multi-provider support for OpenAI-compatible endpoints, Anthropic, Hugging Face, and failover bundles

compression + memory so long tasks can be shrunk into a compact working context

agent traces that can be exported into training data

benchmark + harness runs so the system is testable instead of vibes-based

That last bit matters…