Menu

#Eval

29 posts

Feed·
20 of 29 posts
Use your eval as a CI gate with Agents CLI
📰
0Read More

Use your eval as a CI gate with Agents CLI

DEV Community: cicd·Elia Secchi·5 days ago
#xNIL4v7d
#devtocicd#devto#dev#eval#agents

How to make a quality drop fail your build with agents-cli eval run and one threshold, for anyone who ships an agent from a pipeline.

15s
0Read More

[self-promotion] We started Creeks, a data company for coding, clinical and education data. Looking for labs that need data, and experts who want to help curate it

#datasets#redditdatasets#creeks#eval#tasks

Hey all, we just launched Creeks (https://creeks.ai). We build and sell training data, eval data and RL environments for AI teams. Three domains for now: Code: multi-file tasks in real repos with tests, agentic coding trajectories, held-out eval sets…

15s
Is your new search actually better? Label gaps, LLM judge panels, and paired statistics for offline search evaluation
📰
0Read More

Is your new search actually better? Label gaps, LLM judge panels, and paired statistics for offline search evaluation

DEV Community: python·Naser Sadrmanesh·16 days ago
#V3TykBMQ
#devtopython#devto#dev#judge#queries

An engineering walk through llm-search-eval: why unlabeled results bias comparisons, how judge panels and agreement stats keep LLM labels honest, and why every delta gets a paired test and interval.

15s
[Agent Platform in Practice (5)] Building an AI Agent Quality Flywheel: Automated Benchmarking for LINE AI Business Card King wi
📰
0Read More

[Agent Platform in Practice (5)] Building an AI Agent Quality Flywheel: Automated Benchmarking for LINE AI Business Card King wi

DEV Community: cloud·Evan Lin·24 days ago
#IdgXbY90
#devtocloud#devto#dev#eval#agents

Recap In the first four articles of the "Agent Platform in Action" series, we have...

15s
GitHub - End2End-Diffusion/diffusion-bench: Towards Holistic evaluation of Generative Diffusion Transformers!
🖼️
41Read More

GitHub - End2End-Diffusion/diffusion-bench: Towards Holistic evaluation of Generative Diffusion Transformers!

Hacker News·5 months ago
#fOgVthDd
#github#imagenet#stage#training#evaluation

Towards Holistic evaluation of Generative Diffusion Transformers! - End2End-Diffusion/diffusion-bench

15s
193Read More

(How to Write a (Lisp) Interpreter (in Python))

www.norvig.com·5 months ago
#WWciWtyz
#disqus_thread#define#scheme#environment#list

Why does this matter? As Steve Yegge said, "If you don't know how compilers work, then you don't know how computers work." Yegge describes 8 problems that can be solved with compilers (or equally well with interpreters, or with Yegge's typical heavy…

15s
oh-my-agent: skills now measure and optimize their own utility
🖼️
0Read More

oh-my-agent: skills now measure and optimize their own utility

DEV Community: productivity·gracefullight·5 months ago
#vj4mkULz
#dev#slide#skill#skills#eval

Most skill libraries grow by accretion. You add a SKILL.md, it sounds useful, and it lives forever...

15s
Anthropic Message Batching: When 50% Off Is Worth the Latency
🖼️
0Read More

Anthropic Message Batching: When 50% Off Is Worth the Latency

DEV Community·Gabriel Anhaia·7 months ago
#LopdfHQx
#ai#anthropic#python#llm#batch

Anthropic Batches API gives you half-price tokens with a 24h SLA. Here is when it earns its keep, and a Python script that runs 1,000 evals through it.

15s