1. Small Objects at Scale
2. Latency Sensitivity
3. The Need for Directories
I’m skeptical on the last one. They talk about rename performance as being the issue.
I think what they mean is if you use path as the object key, if you rename a directory in the middle of a path, you need rename every object key that uses it.
But to me that is just a poor usage of an object store. You should never “rename” object keys.
Consider how git does it. If you rename a directory and diff it, the underlying object store didn’t rename any key. In fact all the files in the object stores are unchanged. Only the tree file changed, which maps paths to file hashes.
While renames would get faster that way, it would increase latency to do a path to object key look up.
I would like to see how fundamental the requirement to have directories are to AI workflows. I suspect it’s human “but I’m used to it” requirement