Every time you type a query into Google, Elasticsearch, or even VS Code's file search, a chain of algorithms runs beneath: tokenization, stemming, inverted indexing, scoring, ranking. Most developers treat these as black boxes.

I opened the box. SearchLite is a full-text search engine I built from scratch in Python — zero external dependencies — implementing every layer from raw text analysis to BM25-ranked results.

The Problem

I wanted to understand how search actually works . Not "call elasticsearch.search() " understanding, but "here's why 'running' matches 'run' and why document #7 ranks above document #3" understanding.

The goal: build something that handles real queries like python AND "data pipelines" NOT java with proper relevance ranking.

Architecture

Query → Parser → AST → Posting List Operations → Scorer → Ranked Results ↑ Text → Tokenizer → Normalizer → Stemmer → Inverted Index

Enter fullscreen mode

Exit fullscreen mode

The search engine has 16 modules, each handling one piece:

Layer…