←back to thread

Polars Cloud and Distributed Polars now available

(pola.rs)
183 points jonbaer | 1 comments | 04 Sep 25 03:01 UTC | HN request time: 0.341s | source
Show context
robertkoss ◴[04 Sep 25 09:57 UTC] No.45125481[source]▶
Love it!

Still don't get why one of the biggest player in the space, Databricks is overinvesting in Spark. For startups, Polars or DuckDB are completely sufficient. Other companies like Palantir already support bring your own compute.

replies(2): >>45125682 #>>45126949 #
1. whyever ◴[04 Sep 25 10:35 UTC] No.45125682[source]▶
That's a good question! Especially after Frank McSherry's COST paper [1], it's hard to imagine where the sweet spot for Spark is. I guess for Databricks it makes sense to push Spark, since they are the ones who created it. In a way, it's their competitive advantage.

[1]: https://www.usenix.org/system/files/conference/hotos15/hotos...