Every few months, a new "AI research tool" hits the front page of Hacker News or Product Hunt. Almost without exception, they all do the same thing:

Ingest a PDF.

Chunk it into embeddings.

Feed the chunks into a vector database.

Put a conversational chatbot in front of it that summarizes what the authors said.

Here's the fundamental problem: Science is not a collection of static text summaries. Science is an evolving, adversarial debate.

A paper published in 2023 demonstrating a 4x inference speedup for 4-bit quantized language models on Apple Silicon is not an absolute truth. It is a finding contingent on specific hardware, quantization algorithms, precision levels, and thermal conditions.…