Harbor is a framework from the creators of Terminal-Bench for evaluating and optimizing agents and language models. You can use Harbor to:

Evaluate arbitrary agents like Claude Code, OpenHands, Codex CLI, and more.

Build and share your own benchmarks and environments.

Conduct experiments in thousands of environments in parallel through providers like Daytona, Modal, LangSmith, Blaxel, Novita Sandbox, Tensorlake, Runta, and Sail.

Generate rollouts for RL optimization.

Check out the Harbor Cookbook for end-to-end examples and guides.

Installation

or

Example: Running Terminal-Bench-2.0

Harbor is the official harness for Terminal-Bench-2.0 :

harbor run --dataset [email protected] \ --agent claude-code \ --model anthropic/claude-opus-5-5 \ --n-concurrent 4"> export ANTHROPIC_API_KEY= < YOUR-KEY > harbor run --dataset [email protected] \ --agent claude-code \ --model anthropic/claude-opus-5-5 \ --n-concurrent 4

This will launch the benchmark locally using Docker.…