munchable.app/sitemap.xml is 516 URLs this morning. You can open it and count: 461 ingredient answer pages, 38 recipes, 8 condition guides, and 9 pages typed out by hand. Not one of those URLs is produced by a database query, and that is the whole point of this post, because for a long time nearly all of them were.

What it used to be

Munchable had a public product namespace: one page per barcoded product, generated from rows in our own catalogue. Tens of thousands of them. A sitemap of that size does not fit in a file, because the spec caps a single sitemap at 50,000 URLs, so the shape it forces on you is a sitemap index plus chunked child sitemaps.

That meant:

a hand-rolled index route, listing the chunks

dynamic child routes, one per chunk, so the rows could be paged out of Postgres at request time

about 120 lines of XML serialisation, which is to say 120 lines where a stray ampersand in a product name becomes an invalid document

a custom cache header on those routes, because the query behind a…