←back to thread

Learn Prolog Now

(lpn.swi-prolog.org)
226 points rramadass | 1 comments | | HN request time: 0s | source
Show context
disambiguation ◴[] No.45902807[source]
I am once again shilling the idea that someone should find a way to glue Prolog and LLMs together for better reasoning agents.

https://news.ycombinator.com/context?id=43948657

Thesis:

1. LLMs are bad at counting the number of r's in strawberry.

2. LLMs are good at writing code that counts letters in a string.

3. LLMs are bad at solving reasoning problems.

4. Prolog is good at solving reasoning problems.

5. ???

6. LLMs are good at writing prolog that solves reasoning problems.

Common replies:

1. The bitter lesson.

2. There are better solvers, ex. Z3.

3. Someone smart must have already tried and ruled it out.

Successful experiments:

1. https://quantumprolog.sgml.net/llm-demo/part1.html

replies(15): >>45903080 #>>45903178 #>>45903192 #>>45903204 #>>45903228 #>>45903263 #>>45903361 #>>45903376 #>>45903458 #>>45903841 #>>45904155 #>>45904166 #>>45904490 #>>45906435 #>>45907115 #
jodrellblank ◴[] No.45903192[source]
> "4. Prolog is good at solving reasoning problems."

Plain Prolog's way of solving reasoning problems is effectively:

    for person in [martha, brian, sarah, tyrone]:
      if timmy.parent == person:
        print "solved!"
You hard code some options, write a logical condition with placeholders, and Prolog brute-forces every option in every placeholder. It doesn't do reasoning.

Arguably it lets a human express reasoning problems better than other languages by letting you write high level code in a declarative way, instead of allocating memory and choosing data types and initializing linked lists and so on, so you can focus on the reasoning, but that is no benefit to an LLM which can output any language as easily as any other. And that might have been nice compared to Pascal in 1975, it's not so different to modern garbage collected high level scripting languages. Arguably Python or JavaScript will benefit an LLM most because there are so many training examples inside it, compared to almost any other langauge.

replies(6): >>45903331 #>>45903911 #>>45904372 #>>45905822 #>>45906071 #>>45907029 #
1. ux266478 ◴[] No.45905822[source]
Everything you've written here is an invalid over-reduction, I presume because you aren't terribly well versed with Prolog. Your simplification is not only outright erroneous in a few places, but essentially excludes every single facet of Prolog that makes it a turing complete logic language. What you are essentially presenting Prolog as would be like presenting C as a language where all you can do is perform operations on constants, not even being able to define functions or preprocessor macros. To assert that's what C is would be completely and obviously ludicrous, but not so many people are familiar enough with Prolog or its underlying formalisms to call you out on this.

Firstly, we must set one thing straight: Prolog definitionally does reasoning. Formal reasoning. This isn't debatable, it's a simple fact. It implements resolution (a computationally friendly inference rule over computationally-friendly logical clauses) that's sound and refutation complete, and made practical through unification. Your example is not even remotely close to how Prolog actually works, and excludes much of the extra-logical aspects that Prolog implements. Stripping it of any of this effectively changes the language beyond recognition.

> Plain Prolog's way of solving reasoning problems is effectively:

No. There is no cognate to what you wrote anywhere in how Prolog works. What you have here doesn't even qualify as a forward chaining system, though that's what it's closest to given it's somewhat how top-down systems work with their ruleset. For it to even approach a weaker forward chaining system like CLIPS, that would have to be a list of rules which require arbitrary computation and may mutate the list of rules it's operating on. A simple iteration over a list testing for conditions doesn't even remotely cut it, and again that's still not Prolog even if we switch to a top-down approach by enabling tabling.

> You hard code some options

A Prolog knowledgebase is not hardcoded.

> write a logical condition with placeholders

A horn clause is not a "logical condition", and those "placeholders" are just normal variables.

> and Prolog brute-forces every option in every placeholder.

Absolutely not. It traverses a graph proving things, and when it cannot prove something it backtracks and tries a different route, or otherwise fails. This is of course without getting into impure Prolog, or the extra-logical aspects it implements. It's a fundamentally different foundation of computation which is entirely geared towards formal reasoning.

> And that might have been nice compared to Pascal in 1975, it's not so different to modern garbage collected high level scripting languages.

It is extremely different, and the only reason you believe this is because you don't understand Prolog in the slightest, as indicated by the unsoundness of essentially everything you wrote. Prolog is as different from something like Javascript as a neural network with memory is.