PageSift started as the ingestion layer of an AI research platform I was building: something had to fetch web pages and documents and turn them into clean Markdown that an LLM pipeline could actually use. It grew into a self-hosted web scraper, crawler and document parser — Firecrawl- and Jina-compatible API, MCP server, Python/Node SDKs, one Rust binary.
Over the 1.x series I kept doing one thing: every time a piece of the engine turned out to be generally useful, I cut it out and gave it its own repository, its own versioning, and an MIT license. PageSift 1.4 finishes that split — and adds the subsystem the extraction was making room for.
PageSift 1.4
The split, completed. Wave one (1.2.7) turned the base64, PDF, HTML and content crates into thin shims over standalone libraries. Wave two (1.4.0) does the same for the remaining three: pagesift-office → yolowdoc , pagesift-syntax → yolowsyntax , pagesift-compress → yolowzstd .…