We publish each stage of NewsShorts-Bench as a long write-up: the question, the systems we ran, a results table, the method and what broke. Twelve stages in, the tables were only readable as web pages. Nobody could load them into pandas, and nobody could cite them properly.
So we archived the results as a dataset with a DOI. It took one afternoon. Here is the whole process, because most of it applies to any project that publishes numbers in Markdown.
1. Export from the pages you already publish
Our write-ups are MDX files with the results in frontmatter (name, value, confidence interval, extra columns). The rule we kept: the dataset must be generated from the same files the site renders , never retyped. If a number changes, both change.
import csv , glob , yaml
for path in sorted ( glob . glob ( "content/benchmarks/*.mdx" ) ) : fm = yaml . safe_load ( open ( path , encoding = "utf-8" ) . read ( ) . split ( "---" ) [ 1 ] ) keys = [ c [ "key" ] for c in fm .…