←back to thread

FireDucks: Pandas but Faster

(hwisnu.bearblog.dev)
374 points sebg | 2 comments | 14 Nov 24 11:48 UTC | HN request time: 0.552s | source
Show context
omnicognate ◴[20 Nov 24 12:02 UTC] No.42193108[source]▶
> Then came along Polars (written in Rust, btw!) which shook the ground of Python ecosystem due to its speed and efficiency

Polars rocked my world by having a sane API, not by being fast. I can see the value in this approach if, like the author, you have a large amount of pandas code you don't want to rewrite, but personally I'm extremely glad to be leaving the pandas API behind.

replies(1): >>42193246 #
1. ralegh ◴[20 Nov 24 12:19 UTC] No.42193246[source]▶
I personally found the polars API much clunkier, especially for rapid prototyping. I use it only for cemented processes where I could do with speed up/memory reduction.

Is there anything specific you prefer moving from the pandas API to polars?

replies(1): >>42193784 #
2. benrutter ◴[20 Nov 24 13:31 UTC] No.42193784[source]▶
Not OP but the ability to natively implement complex groupby logic is a huge plus for me at least.

Say you want to take an aggergation like "the mean of all values over the 75th percentile" algonside a few other aggregations. In pandas, this means you're gonna be in for a bunch of hoops and messing around with stuff because you can't express it via the api. Polars' api lets you express this directly without having to implement any kind of workaround.

Nice article on it here: https://labs.quansight.org/blog/dataframe-group-by