Menu

1 / 2

Give the Take-Home a Scoreboard

DEV Community: ai·Morgan Zhou·7 days ago
#RYmB8fzw
#devtoai#devto#dev#probe#probes
Reading 0:00
15s threshold

It is Thursday night and there are three take-homes in your review queue. Each one is a repository, a README, and a paragraph that begins "I'm confident this is production ready." You have twenty minutes per candidate. Within five minutes you are scrolling, and within ten you are grading vibes.

The fix is not a longer rubric document. It is a smaller prompt. Ask the candidate to ship a scoreboard: a script that grades their own submission against probes they wrote, in a form you can re-run on your machine without their laptop, their shell history, or their encouragement.

The prompt that makes grading possible

Keep the implementation small enough that it does not dominate the task. A rolling-window counter over a JSONL event stream is plenty: read {"t": <int seconds>, "key": "a"} lines, emit one line per event with allowed: true|false , honour --window and --limit . Two hours of work, maybe three.

Then add the second half of the prompt, which is the part that matters:

Ship scoreboard.py .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.