←back to thread

From S3 to R2: An economic opportunity

(dansdatathoughts.substack.com)
274 points dangoldin | 1 comments | 02 Nov 23 19:15 UTC | HN request time: 0.419s | source
Show context
thedaly ◴[02 Nov 23 20:35 UTC] No.38119756[source]▶
> In fact, there’s an opportunity to build entire companies that take advantage of this price differential and I expect we’ll see more and more of that happening.

Interesting. What sort of companies can take advantage of this?

replies(3): >>38120302 #>>38120310 #>>38121409 #
1. dangoldin ◴[02 Nov 23 21:19 UTC] No.38120310[source]▶
Author here but some ideas I was thinking about: - An open source data pipeline built on top of R2. A way of keeping data on R2/S3 but then having execution handled in Workers/Lambda. Inspired by what https://www.boilingdata.com/ and https://www.bauplanlabs.com/ are doing. - Related to above but taking data that's stored in the various big data formats (Parquet, Iceberg, Hudi, etc) and generating many more combinations of the datasets and choose optimal ones based on the workload. You can do this with existing providers but I think the cost element just makes this easier to stomach. - Abstracting some of the AI/ML products out there and choosing best one for the job by keeping the data on R2 and then shipping it to the relevant providers (since data ingress to them is free) for specific tasks. -