Menu

How I Fixed Two Hidden Bugs in wreq While Updating My Search Engine
📰

How I Fixed Two Hidden Bugs in wreq While Updating My Search Engine

DEV Community: rust·sandeep mudhi raj·25 minutes ago
#KO0XYEGN
#devtorust#devto#dev#wreq#cookie
Reading 0:00
15s threshold

Recently I contributed to wreq, a fast Rust HTTP library. It started while I was updating my open source Rust search engine SearchWala and testing some of its core libraries. Since SearchWala relies on wreq to fetch web pages, I decided to check the library from a security and reliability angle. The main engine for browser profiles and network requests worked great, but when I looked closely at the cookie and connection layers, I found two real problems.

Cookie security and memory cleanup. Websites should not be allowed to set cookies for broad public domains like .com or .co.uk because that can leak data across unrelated sites. I added checks to block these broad cookies. Also, in a long running app like a search engine, expired cookies were piling up in RAM and every request scanned the full cookie list. Now expired cookies get cleared automatically to free memory, and lookups only check relevant domains instead of scanning everything.

Reusing connections on redirects.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.