Most active commenters
  • pjmlp(5)
  • CyberDildonics(5)

Sparrow: C++20 Idiomatic APIs for the Apache Arrow Columnar Format

(github.com)
36 points tanelpoder | 18 comments | 02 Sep 25 16:09 UTC | HN request time: 2.214s | source | bottom
1. mgaunard ◴[06 Sep 25 09:49 UTC] No.45147930[source]▶
The official arrow implementation is already in C++11, not sure what the value proposition of this is.
replies(2): >>45148047 #>>45148301 #
2. jcelerier ◴[06 Sep 25 10:13 UTC] No.45148047[source]▶
from a quick glance, https://man-group.github.io/sparrow/basic_usage.html looks infinitely more like a library I want to use than https://arrow.apache.org/cookbook/cpp/basic.html
3. levzettelin ◴[06 Sep 25 10:24 UTC] No.45148093[source]▶

  // You are responsible for releasing the structure in the end
  arrow_array.release(&arrow_array);
This doesn't look like RAII. How is this idiomatic for C++20? Why do you have to pass a pointer to "this" again as an explicit argument.
replies(2): >>45148279 #>>45149732 #
4. rfoo ◴[06 Sep 25 11:08 UTC] No.45148279[source]▶
This is the extracted Arrow C data interfaces as documented in https://arrow.apache.org/docs/format/CDataInterface.html

It's not how you interact with the data in your own C++ code, it's for passing this data to other in-process consumers (libraries etc). While in the example it calls the release function, this is usually just passed to a downstream consumer and it's their responsibility to call it.

I agree that having such an example as the first one is confusing. Given that a large part of the point of Apache Arrow is passing data columnar data between libraries in different languages in memory, it makes some sense.

replies(1): >>45150298 #
5. rfoo ◴[06 Sep 25 11:13 UTC] No.45148301[source]▶
<rant>The official Arrow C++ implementation is just ergonomic warts, full of `const std::shared_ptr<T>&` bs. Trying to use it to manipulate data always give me headache telling apart WTH is an Array, ArrayData, Buffer, and the typed Array interfaces are barely usable. The original official Rust port inherited all the mis-designs too. On the Rust side someone created arrow2 [0] to fix it.</rant>

And I'm glad there's a good C++ impl too.

[0] https://github.com/jorgecarleitao/arrow2

replies(1): >>45153184 #
6. pjmlp ◴[06 Sep 25 14:46 UTC] No.45149732[source]▶
RAII predates C++98, I was already used to it in Turbo C++ for MS-DOS, and is pity we need to keep advocating for it as something extraordinary.
replies(2): >>45150286 #>>45152465 #
7. CyberDildonics ◴[06 Sep 25 15:45 UTC] No.45150286{3}[source]▶
This doesn't have anything to do with what they said, they didn't say RAII was new.
replies(1): >>45150811 #
8. CyberDildonics ◴[06 Sep 25 15:46 UTC] No.45150298{3}[source]▶
It's not how you interact with the data in your own C++ code, it's for passing this data to other in-process consumers (libraries etc). While in the example it calls the release function, this is usually just passed to a downstream consumer and it's their responsibility to call it.

This seems like a strange rationalizations when you don't need to have explicit release to be able to pass it to something else.

9. pjmlp ◴[06 Sep 25 16:43 UTC] No.45150811{4}[source]▶
Might be misunderstood by others not skilled in C++ when reading,

> This doesn't look like RAII. How is this idiomatic for C++20?

replies(1): >>45151455 #
10. CyberDildonics ◴[06 Sep 25 17:59 UTC] No.45151455{5}[source]▶
You can try to be insulting if you want but if you could explain the connection I think you would have already.
replies(1): >>45152207 #
11. pjmlp ◴[06 Sep 25 19:32 UTC] No.45152207{6}[source]▶
I wasn't.
replies(1): >>45153707 #
12. ender341341 ◴[06 Sep 25 20:11 UTC] No.45152465{3}[source]▶
I think you're partly making the point for them, RAII has been idiomatic C++ since before c++ was standardized. It wasn't even idiomatic c++98 to be missing it, so to be missing it in c++20 library definitely still isn't.
replies(1): >>45155469 #
13. mgaunard ◴[06 Sep 25 21:49 UTC] No.45153184{3}[source]▶
that's because a given Arrow column is actually several arrays of arrays.

Array, ArrayData and Buffer map to different layers of the abstraction.

14. CyberDildonics ◴[06 Sep 25 23:01 UTC] No.45153707{7}[source]▶
You weren't what? Who are you saying "isn't skilled in C++" here and why would that matter?
replies(1): >>45155472 #
15. pjmlp ◴[07 Sep 25 04:47 UTC] No.45155469{4}[source]▶
Indeed, that is the point.
16. pjmlp ◴[07 Sep 25 04:48 UTC] No.45155472{8}[source]▶
Those that by reading that sentence think RAII is something new in C++20.
replies(1): >>45162457 #
17. CyberDildonics ◴[07 Sep 25 21:44 UTC] No.45162457{9}[source]▶
Why would someone who knows what RAII and C++20 are end up thinking that?