The CPython mmap module docs:
https://docs.python.org/3/library/mmap.htmlzero_buffer (CFFI, 2013) https://github.com/alex/zero_buffer/blob/master/zero_buffer....
"Buffers on the edge: Python and Rust" (2022)
https://alexgaynor.net/2022/oct/23/buffers-on-the-edge/ :
> If you have a Python object and want to obtain its buffer, you can do so with memoryview in Python or PyObject_GetBuffer in C. If you’re defining a class and want to expose a buffer, you can do so in Python by… actually you can’t, only classes implemented in C can implement the buffer protocol. To implement the buffer protocol in C, you provide the bf_getbuffer and bf_releasebuffer functions which are called to obtain a buffer from an object and when that buffer is being released, respectively.
iocursor (CPython C API, ~Rust std::io::Cursor)
https://github.com/althonos/iocursor
Arrow Python (C++) > On disk and MemoryMappedFile s:
https://arrow.apache.org/docs/python/memory.html#on-disk-and...
"Apache Arrow: Read DataFrame With Zero Memory" (2020)
https://towardsdatascience.com/apache-arrow-read-dataframe-w...
pyarrow.Tensor: https://arrow.apache.org/docs/python/generated/pyarrow.Tenso...
ONNX is built on protocolbuffers/protobufs (google/protobufs), while Arrow is built on google/flatbuffers.
FlatBuffers
https://en.wikipedia.org/wiki/FlatBuffers :
> It supports “zero-copy” deserialization, so that accessing the serialized data does not require first copying it into a separate part of memory. This makes accessing data in these formats much faster than data in formats requiring more extensive processing, such as JSON, CSV, and in many cases Protocol Buffers. Compared to other serialization formats however, the handling of FlatBuffers requires usually more code, and some operations are not possible (like some mutation operations).