Menu

1 / 2

How to Speed Up Phrase Search with bigram_index

DEV Community·Sergey Nikolaev·6 months ago
#kLZjX0I1
#example#bigram_index#benchmark#phrase#search
Reading 0:00
15s threshold

TL;DR

bigram_index can be used for several purposes, and in this article we focus specifically on phrase-search performance: on the 1M-document benchmark below, bigram_index='all' improved QPS by about 2.9x and cut average phrase-query latency by about 3.2x .

If your main problem is matching xt850 against xt 850 rather than speeding up phrase search, see How to Make xt850 Match xt 850 .

Phrase search can be expensive. Even when a query is short, the engine still has to verify ordering and adjacency, and that work gets more noticeable when:

the individual words are common

the dataset is large

phrase queries are frequent in your workload

That is exactly what bigram_index is for.

What bigram indexing actually does

Normally, a phrase like "noise cancelling headphones" is handled as separate tokens that also need to appear in the right order and next to each other.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.