←back to thread

Learn Prolog Now

(lpn.swi-prolog.org)
226 points rramadass | 4 comments | | HN request time: 0.001s | source
Show context
disambiguation ◴[] No.45902807[source]
I am once again shilling the idea that someone should find a way to glue Prolog and LLMs together for better reasoning agents.

https://news.ycombinator.com/context?id=43948657

Thesis:

1. LLMs are bad at counting the number of r's in strawberry.

2. LLMs are good at writing code that counts letters in a string.

3. LLMs are bad at solving reasoning problems.

4. Prolog is good at solving reasoning problems.

5. ???

6. LLMs are good at writing prolog that solves reasoning problems.

Common replies:

1. The bitter lesson.

2. There are better solvers, ex. Z3.

3. Someone smart must have already tried and ruled it out.

Successful experiments:

1. https://quantumprolog.sgml.net/llm-demo/part1.html

replies(15): >>45903080 #>>45903178 #>>45903192 #>>45903204 #>>45903228 #>>45903263 #>>45903361 #>>45903376 #>>45903458 #>>45903841 #>>45904155 #>>45904166 #>>45904490 #>>45906435 #>>45907115 #
jodrellblank ◴[] No.45903192[source]
> "4. Prolog is good at solving reasoning problems."

Plain Prolog's way of solving reasoning problems is effectively:

    for person in [martha, brian, sarah, tyrone]:
      if timmy.parent == person:
        print "solved!"
You hard code some options, write a logical condition with placeholders, and Prolog brute-forces every option in every placeholder. It doesn't do reasoning.

Arguably it lets a human express reasoning problems better than other languages by letting you write high level code in a declarative way, instead of allocating memory and choosing data types and initializing linked lists and so on, so you can focus on the reasoning, but that is no benefit to an LLM which can output any language as easily as any other. And that might have been nice compared to Pascal in 1975, it's not so different to modern garbage collected high level scripting languages. Arguably Python or JavaScript will benefit an LLM most because there are so many training examples inside it, compared to almost any other langauge.

replies(6): >>45903331 #>>45903911 #>>45904372 #>>45905822 #>>45906071 #>>45907029 #
1. cpill ◴[] No.45904372[source]
Of course it does "reasoning", what do you think reasoning is? From a quick google: "the action of thinking about something in a logical, sensible way". Prolog searches through a space of logical proposition (constraints) and finds conditions that lead to solutions (if one exists).

(a) Trying adding another 100 or 1000 interlocking proposition to your problem. It will find solutions or tell you one doesn't exist. (b) You can verify the solutions yourself. You don't get that with imperative descriptions of problems. (b) Good luck sandboxing Python or JavaScript with the treat of prompt injection still unsolved.

replies(1): >>45906317 #
2. jodrellblank ◴[] No.45906317[source]
Of course it doesn't "do reasoning", why do you think "following the instructions you gave it in the stupidest way imaginable" is 'obviously' reasoning? I think one definition of reasoning is being able to come up with any better-than-brute-force thing that you haven't been explicitly told to use on this problem.

Prolog isn't "thinking". Not about anything, not about your problem, your code, its implementation, or any background knowledge. Prolog cannot reason that your problem is isomorphic to another problem with a known solution. It cannot come up with an expression transform that hasn't been hard-coded into the interpreter which would reduce the amount of work involved in getting to a solution. It cannot look at your code, reason about it, and make a logical leap over some of the code without executing it (in a way that hasn't been hard-coded into it by the programmer/implementer). It cannot reason that your problem would be better solved with SLG resolution (tabling) instead of SLD resolution (depth first search). The point of my example being pseudo-Python was to make it clear that plain Prolog (meaning no constraint solver, no metaprogramming), is not reasoning. It's no more reasoning than that Python loop is reasoning.

If you ask me to find the largest Prime number between 1 and 1000, I might think to skip even numbers, I might think to search down from 1000 instead of up from 1. I might not come up with a good strategy but I will reason about the problem. Prolog will not. You code what it will do, and it will slavishly do what you coded. If you code counting 1-1000 it will do that. If you code Sieve of Eratosthenes it will do that instead.

replies(2): >>45907118 #>>45907212 #
3. hunterpayne ◴[] No.45907118[source]
Its a Horn clause interpreter. Maybe lookup what that is before commenting on it. Clearly you don't have a good grasp of Computer Science concepts or math based upon your comments here. You also don't seem to understand the AI/ML definition of reasoning (which is based in formal logic, much like Prolog itself).

Python and Prolog are based upon completely different kinds of math. The only thing they share is that they are both Turing complete. But being Turing complete isn't a strong or complete mathematical definition of a programming language. This is especially true for Prolog which is very different from other languages, especially Python. You shouldn't even think of Prolog as a programming language, think of it as a type of logic system (or solver).

4. joshmoody24 ◴[] No.45907212[source]
The disagreement you have with the person you are relying to just boils down to a difference in the definition of "reasoning."