←back to thread

Building a Simple Search Engine That Works

(karboosx.net)
279 points freediver | 1 comments | 17 Nov 25 03:52 UTC | HN request time: 0.239s | source
Show context
marginalia_nu ◴[17 Nov 25 09:44 UTC] No.45952174[source]▶
The idea behind search itself is very simple, and it's a fun problem domain that I encourage anyone to explore[1].

The difficulties in search are almost entirely dealing with the large amounts of data, both logistically and in handling underspecified queries.

A DBMS-backed approach breaks down surprisingly fast. Probably perfectly fine if you're indexing your own website, but will likely choke on something the size of English wikipedia.

[1] The SeIRP e-book is a good (free) starting point https://ciir.cs.umass.edu/irbook/

replies(7): >>45952237 #>>45952734 #>>45952769 #>>45952991 #>>45953075 #>>45953286 #>>45954345 #
HelloUsername ◴[17 Nov 25 11:42 UTC] No.45952769[source]▶
I love your https://marginalia-search.com :)
replies(1): >>45952830 #
marginalia_nu ◴[17 Nov 25 11:53 UTC] No.45952830[source]▶
"Building A Complex Search Engine That Works Sometimes"
replies(1): >>45953299 #
1. moffkalast ◴[17 Nov 25 13:19 UTC] No.45953299[source]▶
15% of the time it works every time.