←back to thread

229 points geetee | 1 comments | | HN request time: 0.001s | source
Show context
tgv ◴[] No.45100192[source]
This makes little sense to me. Ontologies and all that have been tried and have always been found to be too brittle. Take the examples from the front page (which I expect to be among the best in their set): human_activity => climate_change. Those are such a broad concepts that it's practically useless. Or disease => death. There's no nuance at all. There isn't even a definition of what "disease" is, let alone a way to express that myxomatosis is lethal for only European rabbits, not humans, nor gold fish.
replies(15): >>45100256 #>>45100368 #>>45100385 #>>45100399 #>>45100512 #>>45100673 #>>45100711 #>>45100804 #>>45101092 #>>45101484 #>>45103042 #>>45103397 #>>45104046 #>>45108308 #>>45114204 #
jiggawatts ◴[] No.45100385[source]
Even more importantly, it's not even a simple probability of death, or a fraction of a cause, or any simple one-dimensional aspect. Even if you can simplify things down to an "arrow", the label isn't a scalar number. At a bare minimum, it's a vector, just like embeddings in LLMs are!

Even more importantly, the endpoints of each such causative arrow are also complex, fuzzy things, and are best represented as vectors. I.e.: diseases aren't just simple labels like "Influenza". There's thousands of ever-changing variants of just the Flu out there!

A proper representation of a "disease" would be a vector also, which would likely have interesting correlations with the specific genome of the causative agent. [1]

Next thing is that you want to consider the "vector product" between the disease and the thing it infected to cater for susceptibility, previous immunity, etc...

A hop, skip, and a small step and you have... Transformers, as seen in large language models. This is why they work so well, because they encode the complex nuances of reality in a high-dimensional probabilistic causal framework that they can use to process information, answer questions, etc...

Trying to manually encode a modern LLM's embeddings and weights (about a terabyte!) is futile beyond belief. But that's what it would take to make a useful "classical logic" model that could have practical applications.

Notably, expert systems, which use this kind of approach were worked on for decades and were almost total failures in the wider market because they were mostly useless.

[1] Not all diseases are caused by biological agents! That's a whole other rabbit hole to go down.

replies(2): >>45100903 #>>45103306 #
1. energy123 ◴[] No.45103306[source]
You're losing interpretability and scrutability, but gaining detail and expressiveness. You have no way to estimate the vectors in a causal framework, all known methods are correlational. You have no clean way to map the vectors to human concepts. Vectors are themselves extremely compressed representations, there is no clear threshold beyond which a representation becomes "proper".