←back to thread

FireDucks: Pandas but Faster

(hwisnu.bearblog.dev)
374 points sebg | 1 comments | | HN request time: 0s | source
Show context
ssivark ◴[] No.42194215[source]
Setting aside complaints about the Pandas API, it's frustrating that we might see the community of a popular "standard" tool fragment into two or even three ecosystems (for libraries with slightly incompatible APIs) -- seemingly all with the value proposition of "making it faster". Based on the machine learning experience over the last decade, this kind of churn in tooling is somewhat exhausting.

I wonder how much of this is fundamental to the common approach of writing libraries in Python with the processing-heavy parts delegated to C/C++ -- that the expressive parts cannot be fast and the fast parts cannot be expressive. Also, whether Rust (for polars, and other newer generation of libraries) changes this tradeoff substantially enough.

replies(2): >>42194643 #>>42194990 #
1. SiempreViernes ◴[] No.42194990[source]
> Based on the machine learning experience over the last decade, this kind of churn in tooling is somewhat exhausting.

Don't come to old web-devs with those complains, every single one of them had to write at least one open source javascript library just to create their linkedin account!