Menu

I am a lawyer and made a small game: an AI summary of a real judgment with a few planted mistakes, and you catch them by highlighting the proof

Reddit r/artificial·u/siddiboyz·9 days ago
#TiPX0yjd
#judgment#catch#wrong#made#summary
Reading 0:00
15s threshold

Legal AI has learned to summarise. But has it learned how lawyers think?

In my experience, AI summaries of judgments read well and still get things wrong, i.e. a holding stated too broadly, or a point the court rejected reported as accepted. So I made "Catch the AI". You read a real judgment, flag each wrong line in the AI's summary, and a catch counts only when you highlight the words of the judgment that prove it wrong.

It is part of "May or Shall", which I built on the idea that human judgment comes first and AI drafting follows.

Disclosure: I built it. https://app.mayorshall.com/play

I would like to hear how many people here catch all the mistakes, and which ones were the hardest to spot.