Menu

Ask for a Ledger, Not a Demo
πŸ“°

Ask for a Ledger, Not a Demo

DEV Community: careerΒ·Morgan ZhouΒ·8 days ago
#mBSBnnYW
#devtocareer#devto#dev#ledger#self
Reading 0:00
15s threshold

The demo is fine. The candidate posts a link, the chat box answers three questions, and the streaming works. Then you ask what happens on request four thousand, and the screen goes quiet.

That silence is the whole interview. Anyone can get a model to answer once. The engineering is in what your service does when the budget stops being theoretical β€” and a take-home is the cheapest place in the world to find out.

Here is a task I keep coming back to, why the free tier is what makes it portable, and the code, rubric, and failure modes that go with it.

The prompt you hand over

Build a small service that answers questions over a fixed corpus. It must call a hosted model, it must deploy somewhere a reviewer can hit with curl, and it must ship a ledger: a machine-readable record of every token it spent, and a written answer to the question what happens when the budget runs out.

Give them a fixed token allotment for the whole exercise and tell them the allotment is part of the problem, not a footnote.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.