←back to thread

354 points misonic | 2 comments | | HN request time: 0.417s | source
Show context
samsartor ◴[] No.42468798[source]
GNNs have been a bit of a disappointment to me. I've tried to apply them a couple times to my research but it has never worked out.

For a long time GNNs were pitched as a generalization of CNNs. But CNNs are more powerful because the "adjacency weights" (so to speak) are more meaningful: they learn relative positional relationships. GNNs usually resort to pooling, like described here. And you can output an image with a CNN. Good luck getting a GNN to output a graph. Topology still has to be decided up front, sometimes even during training. And the nail in the coffin is performance. It is incredible how slow GNNs are compared to CNNs.

These days I feel like attention has kinda eclipsed GNNs for a lot of those reasons. You can make GNNs that use attention instead of pooling, but there isn't much point. The graph is usually only traversed in order to create the mask matrix (ie attend between nth neighbors) and otherwise you are using a regular old transformer. Often you don't even need the graph adjacencies because some kind of distance metric is already available.

I'm sure GNNs are extremely useful to someone somewhere but my experience has been a hammer looking for a nail.

replies(5): >>42468874 #>>42468882 #>>42469313 #>>42469395 #>>42472618 #
1. energy123 ◴[] No.42469395[source]
Google's GraphCast is a GNN: https://deepmind.google/discover/blog/graphcast-ai-model-for...
replies(1): >>42471677 #
2. eperfa ◴[] No.42471677[source]
Google DeepMind's GenCast is based on diffusion: https://deepmind.google/discover/blog/gencast-predicts-weath...

(Partially) Google Research's/DeepMind's NeuralGCM is based on hybrid models using ODEs and learnt physics: https://www.nature.com/articles/s41586-024-07744-y

Microsoft Research's Aurora on vision transformers: https://www.microsoft.com/en-us/research/blog/introducing-au...

Huawei's Pangu Weather is also a 3D transformer I believe https://www.nature.com/articles/s41586-024-07744-y

I just wanted to highlight that there are multiple approaches in use for the same problem / in the same domain, and GNN does not seem to be the most widely used one.