If you chose a photo of socks pointing left, then the nearest neighbors are socks also pointing left.
I’d think the model should focus on the patterns and cut, not which way they are laying for the marketing photo.
replies(1):
The backend is written in Swift, and is hosted on a single Mac Mini. It performs nearest neighbors on the GPU over ~3M product images.
No vector DB, just pure matrix multiplications. Since we aren't just doing approximate nearest neighbors but rather sorting all results by distance, it's possible to show different "variety" levels by changing the stride over the sorted search results.
Nearest neighbors are computed in a latent vector space. The model which produces the vectors is also something I trained in pure Swift.
The underlying data is about 2TB scraped from https://www.shopltk.com/.
All the code is at https://github.com/unixpickle/LTKlassifier