←back to thread

Turbopuffer: Fast search on object storage

(turbopuffer.com)
379 points Sirupsen | 1 comments | 09 Jul 24 14:48 UTC | HN request time: 0.272s | source
Show context
solatic ◴[10 Jul 24 16:19 UTC] No.40928528[source]▶
Is it feasible to try to build this kind of approach (hot SSD cache nodes sitting in front of object storage) with prior open-source art (Lucene)? Or are the search indexes themselves also proprietary in this solution?

Having witnessed some very large Elasticsearch production deployments, being able to throw everything into S3 would be incredible. The applicability here isn't only for vector search.

replies(2): >>40928889 #>>40929482 #
1. francoismassot ◴[10 Jul 24 16:55 UTC] No.40928889[source]▶
If you don't need vector search and have very large Elasticsearch deployment, you can have a look at Quickwit, it's a search engine on object storage, it's OSS and works for append-only datasets (like logs, traces, ...)

Repo: https://github.com/quickwit-oss/quickwit