I've been going down the LSM / databases-on-object-storage rabbit hole and wanted to see for myself why almost nobody runs an HNSW graph straight off a bucket. So I took HelixDB (graph stored as rows in a fork of SlateDB), counted what one cold query does, and compared the shape with what turbopuffer says it does.
Setup: first 100k SIFT vectors, fresh node, counting requests to the storage layer. Local disk with 20 ms injected per request, so not real S3.
367 requests per query at 71% recall, 1,086 at 97%
never more than 3 in flight
24 s per query (requests × delay, within 6%)
fully cached: 29 ms, zero requests
It's what you'd expect, just worse than I guessed: the walk learns the next neighbor list from the read it's waiting on, so the GETs can't be parallelized, and at ~80 ms per real S3 read the chain is the whole latency. HelixDB's own commit says it: "Every HNSW hop became a serial S3 GET".…