An AI-written code detector can look like it's working while it checks your code with the wrong model. Master the Interview , an AI technical interviewer I built ( source ), labels a candidate's code as AI- or human-written. Reviewing it later, I found its API scored Python and Java with the models trained on JavaScript . The cause was one block at the end of the training script. It raised no error, and the output still looked confident. Its line-by-line evidence was partly made up by the code itself , and the dataset was too small for any accuracy figure to mean much. Here is what went wrong and what I'd ship instead.
How does the detector work?
The setup is a common one for text classification:
Training data: folders of AI-written and human-written files per language.…