Use Prolog to improve LLM's reasoning

Oh, cool. Mind if I pick your brain a bit?

Recently, there was an HN post[0] of a paper that makes a case against pure logic languages in favor of "functional logic" ones, which they exhibit with Curry[1]. The setup argument is that Prolog's specs backtracking, which strongly downlimits it from full SLD resolution, causing fatally sharp edges in real world usage.

Being fairly naive to the paradigm, my interpretation is that writing real Prolog programs involves carefully thinking about and controlling the resolution algorithm, which feels very different than straight knowledge declaration. I believe cut/0 is the go-to example. Is that your experience with Prolog in practice?

The real meat of the paper, however, is in its case that functional logic languages fully embed Prolog with almost 1-to-1 expressivity, while also providing more refined tools for externalizing knowledge about the intended search space of solutions.

Thoughts? How are you using Prolog, logic, or constraint programming? What languages and tooling in this arena do you reach for? What is some of your most hard-earned knowledge? Any lesser-known, but golden, websites, books, or materials you'd like to share?

Cheers!

[0]:https://news.ycombinator.com/item?id=41816545

[1]:https://www.curry-language.org/

But what I actually find is the more time spent in Prolog, the more natural it is to express things in a way that is clear, logical and performant. As with any language/paradigm, there are a few gotchas to be experienced. But generally speaking, SLD resolution has never once been an obstacle (in the past 2 years) of coding.

The general execution model of Prolog is pretty simple. The lack of functions actually makes meta-programming much clearer and simpler. A term is just data, unless it's stated as a goal. It's only a valid goal if you've already defined its meaning.

So I'd be concerned that Curry gives up the simplicity of Prolog's execution model, and ease of meta-programming. I struggle with the lack of types in Prolog, but also know I can (at least in theory) use Prolog to solve correctness problems in Prolog code.

I'm currently using SWI-Prolog. Performance is excellent, it has excellent high-level concurrency primitives[0] (when was the last time you pegged all your cores solving a problem?), and many libraries. I might be one of the few people who has committed to using the integrated editor (PceEmacs) despite being a Vim person. PceEmacs is just too good at syntax highlighting and error detection.

At the same time, I'm a huge fan of Markus Triska. His Youtube[1] stuff is mind-expanding (watch all of it, even if you never write Prolog). He has an excellent book online[2]. I admire the way he explains and advances pure monotonic Prolog, and I appreciate the push for ISO conformance and his support for Prologs that that do the same (SWI is not on that list).

If you want to learn Prolog, watch all of Markus Triska's videos, read his book, and learn what Prolog could be in a perfect world. Then download SWI-Prolog, and maybe break some rules while getting things done at a blazing speed. Eventually you'll gravitate to what makes sense for you.

The Art of Prolog is a classic "must have". Clause and Effect is a good "hit the ground running" (on page 70 you're into symbolic differentiation via term rewriting).

0 https://www.swi-prolog.org/pldoc/man?section=thread

1 https://www.youtube.com/@ThePowerOfProlog

2 https://www.metalevel.at/prolog

1. If you find yourself straying too often from coding in relations, and instead coding in instructive steps, you're going to end up with problems.

2. Use DCGs to create a DSL for any high level operations performed on data structures. The bi-directionality of Prolog's clauses means you can use this DSL to generate an audit trail of "commands executed" when Prolog solves a problem for you, but you can also use the audit trail and modify it to execute those commands on other data.

Still checking every now and then if SICStus has open sourced. I used Prolog daily during my PhD and SICStus had such nice features. E.g. it could raise an exception when no more heap space can be allocated or when a 'call' would not finish within a given time. These features made it much easier to use Prolog in real-world systems (this was a parsing system and when parsing a very large corpus, this was highly preferable over simply crashing the interpreter).

Maybe things have changed, but this wasn't possible with SWI at the time. Even worse, most C extensions would use malloc directly, making it impossible to track allocations done by extensions.

I figured it would be a good introduction to prolog, but to date there doesn't seem to be any prolog interpreter that lets me copy the things in the book to play with them?

Also DCGs for high level operations? Do you mean "use DCGs to parse strings that contain instructions" or do you parse things other than strings with DCGs? I'm assuming you take the parsed instructions and run them through some kind of interpreter that does the execution and audit trail.

>> How do you debug DCGs? I get "false." instead of "syntax error at line 23", which is unacceptable for bigger inputs.

You need to include exception handling in your DCG rules. For example, in Prolog-like pseudocode:

  pink_apples([A|As]) --> [A], { red_apple(A), throw(error(type_error(pink_apple,red_apple),_)) }.

  % Raises type error:
  ERROR: Type error: `pink_apple' expected, found `red_apple' (an atom)
  ERROR: In:
  ERROR:   [12] throw(error(type_error(pink_apple,red_apple),_10070))

https://www.swi-prolog.org/pldoc/man?section=exception

DCGs parse lists, not strings, as such. So the input can be anything you can put in the form of a list.

If what Prolog is doing you consider incorrect, _make it_ incorrect.

DCGs can be used to convert any data structure to a sequence. Actually, they are capable of any graph to graph , so they could produce a sequence of commands.

The oft-cited Markus Triska has some great work on this:

  https://youtu.be/vdabv9EkYrY?feature=shared

  https://www.metalevel.at/zurg/

See https://github.com/mthom/scryer-prolog/blob/master/src/lib/p...

That was my concern with the paper listed above also. Functional syntax is to my mind needlessly over-complicated. Types are useful but you can roll your own if you need them, and Prolog makes that easy enough.

>> I'm currently using SWI-Prolog. Performance is excellent, it has excellent high-level concurrency primitives[0] (when was the last time you pegged all your cores solving a problem?), and many libraries. I might be one of the few people who has committed to using the integrated editor (PceEmacs) despite being a Vim person. PceEmacs is just too good at syntax highlighting and error detection.

Hah! Hello fellow PCEmacs >> vim user :D

Markus Triska's stuff is good and he's undeniably an expert in Prolog programming, I mean duh, but he and a couple of others have needlessly caused a rift in the Prolog community (mainly between themselves and everyone else) by being so stroppy about the ISO non-conformance of SWI-Prolog. I hope Markus is reading this. The Prolog community is very small and dwindling and we can't afford such drama. We need more Prolog compilers, yes, ISO conformance is good, yes, but SWI-Prolog is a robust and battle-tested implementation and it is the ISO standard that should be leaning on its experience of being a real-world Prolog used by real programmers and not just the other way around.

There used to be a free (though proprietary) version of Quintus, the predecessor of SICStus. It might still be around somewhere if you look hard enough.

I also sympathize. "false" as the default failure mode is a challenge with Prolog. Most Prologs I've used have good debugging/stepping features (see spy and trace predicates), logical debugging of pure monotonic Prolog can often help (explained by Markus Triska), you can easily write (use existing) meta predicates that assert a called predicate must not fail otherwise throw an exception. For example: here the ./ is supposed to look like a checkmark. So `./ true.` is true. `./ false` throws an exception.

  :- op(920, fy, './').
  :- meta_predicate './'(0).
  './'(X) :- call(X) *-> true ; prolog_debug:assertion_failed(fail, X).

That made my day! Hello!

SICStus, GNU Prolog, XSB, Ciao, Scryer, Traella, Tau but does not mention at all of SWI-Prolog. I remember using SWI-Prolog back in the day, did it somehow fall out favor, or is there some animosity between implementation and Markus is just not in the SWI-Prolog camp?

It's really surprising to see someone at his level, a software reliability researcher no less, retain that level of passion about something. Usually folks with that level of experience are really grumpy and lose the evangelistic flair. I tend to maintain that level of evangelism about things I'm passionate for but no one would ever accuse me of being a "reliability researcher", lol.

In fact, I was/am a hardcore lisp hacker, and never in my life did I think I'd find something that would even come close to lisp for me. because it's FUN! but I'll be damned if Prolog isn't turning out to be even more fun in ways I didn't expect.

In fact the only thing I really miss about lisp at this point is structural editing (paredit and such).

And it's not like it's either/or, you can use both. But learning pure Prolog is really delightful if you follow the breadcrumbs (gold nuggets, honestly) Markus left.

If it doesn't kill you, that is!

You don't have to take my word for it, you can just see what he says and does. But I've seen nearly every one of his videos and blog posts, read through many previous comments on HN, there's not a single disparaging remark in any of them. In fact he even makes his philosophy clear in this post [1]:

  Things that I, at least, always do without even thinking about it. For instance, when I mention Scryer Prolog in Internet discussions, I always try to mention at least one other Prolog system too in a flattering way, out of respect and admiration for other systems and also to encourage more cooperation between systems.

As you've seen from the comments though, SWI is an extremely successful, established system. Nearly every book and example you will read uses SWI. It has great libraries, great IDEs, FFI, embedding, support, documentation, all kinds of great stuff -- SWI is already REALLY successful, and for very good reasons!

So my guess is just trying to give a voice to the little guys who are up and coming and are in line philosophically with his beliefs about the language and open source. I personally would describe that as "support", I don't think I would use the word "animosity".

[1]: https://github.com/mthom/scryer-prolog/discussions/2605

I like the analogy!

> So my guess is just trying to give a voice to the little guys who are up and coming and are in line philosophically with his beliefs about the language and open source.

That makes perfect sense. Thanks for answering! It's an interesting insight into the world of "Prologs" to a complete outsider. I had only used SWI-Prolog in the university and remember it having a variety of modules, web development libraries and other such things, so at least superficially to a novice, that was pretty impressive.