←back to thread

Using gRPC for (local) inter-process communication (2021)

(www.mpi-hd.mpg.de)
121 points zardinality | 4 comments | 20 Nov 24 13:42 UTC | HN request time: 0.951s | source
Show context
pengaru ◴[20 Nov 24 16:34 UTC] No.42195597[source]▶
There's a mountain of grpc-centric python code at $dayjob and it's been miserable to live with. Maybe it's less awful in c/c++, or at least confers some decent performance there. In python it's hot garbage.
replies(5): >>42195747 #>>42196231 #>>42196568 #>>42196845 #>>42197041 #
andy_ppp ◴[20 Nov 24 16:47 UTC] No.42195747[source]▶
Strongly agree, it’s has loads of problems, my least favourite being the schema is not checked in the way you might think, there’s not even a checksum to say this message and this version of the schema match. So when there’s old services/clients around and people haven’t versioned their schema’s safely (there was no mechanism for this apart from manually checking in PRs) you can get gibberish back for fields that should contain data. It’s basically just a binary blob with whatever schema the client has overlaid so debugging is an absolute pain. Unless you are Google scale use a text based format like JSON and save yourself a lot of hassle.
replies(3): >>42195995 #>>42196041 #>>42196375 #
jeffbee ◴[20 Nov 24 17:42 UTC] No.42196375[source]▶
There is an art to having forwards and backwards compatible RPC schemas. It is easy, but it is surprisingly difficult to get people to follow easy rules. The rules are as follows:

  1) Never change the type of a field
  2) Never change the semantic meaning of a field
  3) If you need a different type or semantics, add a new field
Pretty simple if you ask me.
replies(1): >>42196428 #
1. andy_ppp ◴[20 Nov 24 17:47 UTC] No.42196428[source]▶
If I got to choose my colleagues this would be fine, unfortunately I had people who couldn’t understand eventual consistency. One of the guys writing Go admitted he didn’t understand what a pointer was etc. etc.
replies(1): >>42197914 #
2. lrem ◴[20 Nov 24 20:43 UTC] No.42197914[source]▶
How does JSON protect you from that?
replies(1): >>42198466 #
3. andy_ppp ◴[20 Nov 24 21:48 UTC] No.42198466[source]▶
People understand JSON fairly commonly as they can see what is happening in a browser or any other system - what is the equivalent for GRPC if I want to do console.log(json)?

GRPC for most people is a completely black box with unclear error conditions that are not as clear to me at least. For example what happens if I have an old schema and I'm not seeing a field, there's loads of things that can be wrong - old services, old client, even messages not being routed correctly due to networking settings in docker or k8s.

Are you denying there is absolutely tones to learn here and it is trickier to debug and maintain?

replies(1): >>42227849 #
4. lrem ◴[24 Nov 24 13:31 UTC] No.42227849{3}[source]▶
I'd go with `cerr << request.DebugString() << '...' << response.DebugString();`, preferably with your favourite logger instead of `stdout`. My browser does the equivalent for me just fine, but that required an extension.

I buy the familiarity argument, but I usually don't see the wire format at all. And maintenance-wise, protobufs seem easier to me. But that's because, e.g., someone set up a presubmit for me that yells at me if my change isn't backwards compatible. That's kind of hard to do if you don't have a formal specification of what goes into your protocol.