Menu

yolow64: a Base64 codec for Rust that outruns the incumbents — extracted from PageSift
📰

yolow64: a Base64 codec for Rust that outruns the incumbents — extracted from PageSift

DEV Community: rust·ゴエンチョウ·3 days ago
#CXhw92Qh
#devtorust#devto#dev#base64#decode
Reading 0:00
15s threshold

While building PageSift (a Rust web-content engine for LLM pipelines), I kept hitting Base64 in places where throughput and allocation control actually matter: browser screenshots, MCP resources, proxy authentication. The codec that grew inside PageSift is now its own MIT-licensed library: yolow64 .

Runtime-dispatched SIMD: AVX2 (with an assembly bulk decode loop on Linux x86-64), NEON on AArch64, scalar fallback; runtime detection is cached.

Zero runtime dependencies, std only, Rust 1.89+.

Slice / append / owned-output APIs, validation without decoding, streaming decode, and a one-shot data-URL builder.

Whitespace-tolerant mode ignores exactly space/tab/CR/LF in a single pass — built for MIME-style input.

Measured on an i7-10700 (same process, three full runs, medians of seven-round medians; full method, charts and raw CSVs in the repo):

Strict mode at 4 KiB–4 MiB: ~6.5–9× faster than the base64 crate, ~1.7–2.5× faster than base64-simd , encode and decode.

CRLF-tolerant decode: ~4× faster than…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.