←back to thread

276 points Fendy | 1 comments | | HN request time: 0.199s | source
Show context
simonw ◴[] No.45170289[source]
This is a good article and seems well balanced despite being written by someone with a product that directly competes with Amazon S3. I particularly appreciated their attempt to reverse-engineer how S3 Vectors work, including this detail:

> Filtering looks to be applied after coarse retrieval. That keeps the index unified and simple, but it struggles with complex conditions. In our tests, when we deleted 50% of data, TopK queries requesting 20 results returned only 15—classic signs of a post-filter pipeline.

Things like this are why I'd much prefer if Amazon provided detailed documentation of how their stuff works, rather than leaving it to the development community to poke around and derive those details independently.

replies(5): >>45171116 #>>45171985 #>>45172432 #>>45177278 #>>45180236 #
1. apwell23 ◴[] No.45180236[source]
That would increase surface area of the abstraction they are trying to expose. This is not a case of failure to document.

One should only "poke around" an abstraction like this for fun and curiosity and not with intention of putting the finding to real use.