The demo is fine. The candidate posts a link, the chat box answers three questions, and the streaming works. Then you ask what happens on request four thousand, and the screen goes quiet.
That silence is the whole interview. Anyone can get a model to answer once. The engineering is in what your service does when the budget stops being theoretical β and a take-home is the cheapest place in the world to find out.
Here is a task I keep coming back to, why the free tier is what makes it portable, and the code, rubric, and failure modes that go with it.
The prompt you hand over
Build a small service that answers questions over a fixed corpus. It must call a hosted model, it must deploy somewhere a reviewer can hit with curl, and it must ship a ledger: a machine-readable record of every token it spent, and a written answer to the question what happens when the budget runs out.
Give them a fixed token allotment for the whole exercise and tell them the allotment is part of the problem, not a footnote.β¦