indexation is pretty slow

[setop]

I built an index of 27k entries that are short sentences (nine English words average).

The search is pretty quick : 300ms in CLI, 600ms in the browser.

But building the index has been very long : 15 minutes on a core i7 CPU. Means 30 entries / seconds.

I've been doing the same exercise with other search engine also written is Rust and it was running at 2700 entries / seconds.

Any idea of what cost so much ? Any ways to optimize it ?

[jameslittle230]

Hi! I'm sure there are optimizations that could be made to the indexer, but I haven't spent much time optimizing the indexer itself (or investigating what could be optimized) -- I've instead focused more on build artifact size and search speed.

I'll leave this issue open for a bit to see if any optimization ideas crop up.

-James