←back to thread

FireDucks: Pandas but Faster

(hwisnu.bearblog.dev)
374 points sebg | 2 comments | 14 Nov 24 11:48 UTC | HN request time: 0s | source
Show context
rich_sasha ◴[20 Nov 24 11:56 UTC] No.42193043[source]▶
It's a bit sad for me. I find the biggest issue for me with pandas is the API, not the speed.

So many foot guns, poorly thought through functions, 10s of keyword arguments instead of good abstractions, 1d and 2d structures being totally different objects (and no higher-order structures). I'd take 50% of the speed for a better API.

I looked at Polars, which looks neat, but seems made for a different purpose (data pipelines rather than building models semi-interactively).

To be clear, this library might be great, it's just a shame for me that there seems no effort to make a Pandas-like thing with better API. Maybe time to roll up my sleeves...

replies(22): >>42193093 #>>42193139 #>>42193143 #>>42193309 #>>42193374 #>>42193380 #>>42193693 #>>42193936 #>>42194067 #>>42194113 #>>42194302 #>>42194361 #>>42194490 #>>42194544 #>>42194670 #>>42195628 #>>42196720 #>>42197192 #>>42197489 #>>42198158 #>>42199832 #>>42200060 #
1. martinsmit ◴[20 Nov 24 12:01 UTC] No.42193093[source]▶
Check out redframes[1] which provides a dplyr-like syntax and is fully interoperable with pandas.

[1]: https://github.com/maxhumber/redframes

replies(1): >>42195287 #
2. otsaloma ◴[20 Nov 24 16:10 UTC] No.42195287[source]▶
Building on top of Pandas feels like you're only escaping part of the problems. In addition to the API, the datatypes in Pandas are a mess, with multiple confusing (and none of them good) options for e.g. dates/datetimes. Does redframes do anything there?