←back to thread

A Gentle Introduction to Graph Neural Networks (2021)

(distill.pub)
354 points misonic | 1 comments | 20 Dec 24 04:10 UTC | HN request time: 0s | source
Show context
cherryteastain ◴[20 Dec 24 10:07 UTC] No.42469706[source]▶
There are a lot of papers using GNNs for physics simulations (e.g. computational fluid dynamics) because the unstructured meshes used to discretize the problem domain for such applications map very neatly to a graph structure.

In practice, every such mesh/graph is used once to solve a particular problem. Hence it makes little sense to train a GNN for a specific graph. However, that's exactly what most papers did because no one found a way to make a GNN that can adjust well to a different mesh/graph and different simulation parameters. I wonder if there's a breakthrough waiting just around the corner to make such a generalization possible.

replies(2): >>42470241 #>>42470602 #
magicalhippo ◴[20 Dec 24 11:31 UTC] No.42470241[source]▶
Naive question:

Words in sentences kinda forms graphs, referencing other words or are leafs being referenced, both inside sentences and between sentences.

Given the success of the attention mechanism in modern LLMs, how well would they do if you trained a LLM to process an actual graph?

I guess you'd need some alternate tokenizer for optimal performance.

replies(4): >>42470363 #>>42470817 #>>42472814 #>>42474565 #
1. 6gvONxR4sf7o ◴[20 Dec 24 20:13 UTC] No.42474565[source]▶
That's the kind of thing that I could imagine could dramatically speed up certain tasks, but not enable particularly new abilities. A ton of the challenge is converting a sequence into a graph. So if you need a huge clever model to turn a sequence into a graph, then your potential gain downstream is either a) in making certain computationally hard queries easier, or b) answering tons and tons of followup queries dramatically faster (enough to make the initial graphification overhead okay).

For (a), any imperfections in the graphification make the problem super hard and researchy.