#Correctness

Systems that generate GPU kernels with language models report high correctness rates. Those rates come from a single loose test: run the kernel on a few random inputs at one fixed shape and accept it if the output is close to a reference.…

This somewhat widely cited post (I keep seeing it cited, anyway) suggests that dynamic languages and/or languages that represent things more concisely are more token efficient. It seems to be cited enough that LLM search results agree.…

Most people benchmark inference engines on throughput. Tokens per second, batch size limits, latency...

From Dev.to - java: Splitting Correctness from Throughput: A Hybrid Approach to XLSX Streaming Writes

We tend to assume that if every part of a system behaves correctly, the system itself will behave correctly. That assumption is deeply embedded in how we design, test, and operate software.…