←back to thread

Show HN: Apache Fory Rust – 10-20x faster serialization than JSON/Protobuf

(fory.apache.org)
66 points chaokunyang | 1 comments | 28 Oct 25 17:58 UTC | HN request time: 0s | source

Serialization framework with some interesting numbers: 10-20x faster on nested objects than json/protobuf.

  Technical approach: compile-time codegen (no reflection), compact binary protocol with meta-packing, little-endian layout optimized for modern CPUs.

  Unique features that other fast serializers don't have:
  - Cross-language without IDL files (Rust ↔ Python/Java/Go)
  - Trait object serialization (Box<dyn Trait>)
  - Automatic circular reference handling
  - Schema evolution without coordination

  Happy to discuss design trade-offs.

  Benchmarks: https://fory.apache.org/docs/benchmarks/rust
Show context
no_circuit ◴[28 Oct 25 20:50 UTC] No.45738982[source]▶
Is 4096 types enough for everyone?

https://github.com/apache/fory/blob/fd1d53bd0fbbc5e0ce6d53ef...

replies(1): >>45742318 #
chaokunyang ◴[29 Oct 25 03:33 UTC] No.45742318[source]▶
Probably not for everyone. The current limit of 4096 types could be expanded if there’s a real need — it’s not a hard technical barrier.

I’m curious though: what’s an example scenario you’ve seen that requires so many distinct types? I haven’t personally come across a case with 4,096+ protocol messages defined.

replies(1): >>45750123 #
1. no_circuit ◴[29 Oct 25 17:25 UTC] No.45750123[source]▶
If your binary has a small function set, probably not. But in a use case if you want to proxy/intercept cloud APIs, then something like Google APIs has 34K message types:

    git clone https://github.com/googleapis/googleapis.git
    cd googleapis
    find . -name '*.proto' -and -not -name '*test*' -and -not -name '*example*' -exec grep '^message' {} \; | wc -l
I think this more speaks to the tradeoff of not having an IDL where the deserializer either knows what type to expect if it was built with the IDL file version that defined it, e.g., this recent issue:

https://github.com/apache/fory/issues/2818

But now I do see that the 4096 is just arbitrary:

    If schema consistent mode is enabled globally when creating fory, type meta will be written as a fory unsigned varint of type_id. Schema evolution related meta will be ignored.