Out of curiosity, what's the size of vectors you're using (# of dimensions) and what distance metric are you using? Euclidean?
replies(1):
The backend is written in Swift, and is hosted on a single Mac Mini. It performs nearest neighbors on the GPU over ~3M product images.
No vector DB, just pure matrix multiplications. Since we aren't just doing approximate nearest neighbors but rather sorting all results by distance, it's possible to show different "variety" levels by changing the stride over the sorted search results.
Nearest neighbors are computed in a latent vector space. The model which produces the vectors is also something I trained in pure Swift.
The underlying data is about 2TB scraped from https://www.shopltk.com/.
All the code is at https://github.com/unixpickle/LTKlassifier
The site uses cosine distance. The code itself implements Euclidean distance, but I decided to normalize the vectors last minute out of FUD that some unusually small vectors would appear as neighbors for an abnormal number of examples.