←back to thread

Polars Cloud and Distributed Polars now available

(pola.rs)
183 points jonbaer | 2 comments | 04 Sep 25 03:01 UTC | HN request time: 0s | source
Show context
lvl155 ◴[04 Sep 25 10:33 UTC] No.45125676[source]▶
Polars is certainly better than pandas doing things locally. But that is a low bar. I’ve not had great experience using Polars on large enough datasets. I almost always end up using duckdb. If I am using SQL at the end of the day, why bother starting with Polars? With AI these days, it’s ridiculously fast to put together performant SQLs. Heck you can even make your own grammar and be done with it.
replies(3): >>45126319 #>>45126571 #>>45126595 #
1. sirfz ◴[04 Sep 25 12:39 UTC] No.45126571[source]▶
SQL is definitely easier and faster to compose than any dataframe syntax but I think pandas syntax (via slicing API) is faster to type and in most cases more intuitive but I still use polars for all df-related tasks in my workflow since it's more structured and composable (although needs more time to construct but that's a cost I'm willing to take when not simply prototyping). When in an ipython session, sql via duckdb is king. Also: python -m chdb "describe 'file.parquet'" (or any query) is wonderful
replies(1): >>45126973 #
2. mr_toad ◴[04 Sep 25 13:20 UTC] No.45126973[source]▶
> SQL is definitely easier and faster to compose

Sometimes. But sometimes Python is just much easier. For example transposing rows and columns.