←back to thread

Show HN: Fashion Shopping with Nearest Neighbors

(vibewall.shop)
76 points unixpickle | 1 comments | 15 Mar 25 15:33 UTC | HN request time: 0.218s | source

I made this website with my wife in mind; it makes it possible to browse for similar fashion products over many different retailers at once.

The backend is written in Swift, and is hosted on a single Mac Mini. It performs nearest neighbors on the GPU over ~3M product images.

No vector DB, just pure matrix multiplications. Since we aren't just doing approximate nearest neighbors but rather sorting all results by distance, it's possible to show different "variety" levels by changing the stride over the sorted search results.

Nearest neighbors are computed in a latent vector space. The model which produces the vectors is also something I trained in pure Swift.

The underlying data is about 2TB scraped from https://www.shopltk.com/.

All the code is at https://github.com/unixpickle/LTKlassifier

Show context
abeppu ◴[15 Mar 25 21:39 UTC] No.43375332[source]▶
It's interesting to see how some ideas keep looping around. When I see this I'm reminded of a startup Modista from ~2008 which did visual-similarity for browsing fashion products. Here's a writeup from a point when they did only shoes, but I know they expanded beyond shoes at somepoint: https://thenoisychannel.com/2008/11/05/modista-similarity-br...

I think they died over legal issues with rights to product images.

But in the vibewall demo, I wonder if the embedding is capturing the right similarity concept for this application. E.g. in the results most similar to this men's polo, I see a bunch of henleys, a women's quarter-zip pull-over, a women's full-zip fleece, a men's tank, a women's top with a plunging neckline, even a baby wrap! These are appropriate to be worn by different people in different social contexts and in different seasons. The main visual similarity seems to be that they include human upper bodies on white backgrounds? https://vibewall.shop/?id=c43bc222-e68b-11ef-8208-0242ac1100...

replies(1): >>43375415 #
1. unixpickle ◴[15 Mar 25 21:59 UTC] No.43375415[source]▶
You definitely highlighted a shortcoming of the feature vector model in this case. Indeed it's quite a small model trained on a single Mac for about a week, so it's not very "smart".

I'd expect that this is a problem that could be solved by using larger off the shelf models for image similarity. For this project, I thought it would be cooler to train the model end-to-end myself, but doing so has negative consequences for sure.