←back to thread

Use Prolog to improve LLM's reasoning

(shchegrikovich.substack.com)
379 points shchegrikovich | 7 comments | 13 Oct 24 21:22 UTC | HN request time: 3.08s | source | bottom
Show context
YeGoblynQueenne ◴[17 Oct 24 21:53 UTC] No.41874200[source]▶
That's not going to work. Garbage in - Garbage out is success-set equivalent to Garbage in - Prolog out.

Garbage is garbage and failure to reason is failure to reason no matter the language. If your LLM can't translate your problem to a Prolog program that solves your problem- Prolog can't solve your problem.

replies(3): >>41874322 #>>41875551 #>>41876345 #
Philpax ◴[17 Oct 24 22:10 UTC] No.41874322[source]▶
This is a shallow critique that does not engage with the core idea. Specifying the problem is not the same as solving the problem.
replies(2): >>41874902 #>>41880822 #
YeGoblynQueenne ◴[17 Oct 24 23:34 UTC] No.41874902[source]▶
I've programmed in Prolog for ~13 years and my PhD thesis is in machine learning of Prolog programs. How deep would you like me to go?
replies(3): >>41875047 #>>41875563 #>>41879227 #
1. upghost ◴[18 Oct 24 13:24 UTC] No.41879227[source]▶
can you link your thesis? That sounds awesome!!
replies(1): >>41880711 #
2. YeGoblynQueenne ◴[18 Oct 24 16:08 UTC] No.41880711[source]▶
I can't seem to find it on the web. I don't think it's published online yet.
replies(1): >>41882870 #
3. upghost ◴[18 Oct 24 19:52 UTC] No.41882870[source]▶
that's ok (T_T)

I'll just imagine what it could've been like! I bet it was awesome!

replies(1): >>41884673 #
4. YeGoblynQueenne ◴[19 Oct 24 00:32 UTC] No.41884673{3}[source]▶
It's a PhD thesis... It was put to me that nobody will ever read it and nobody cares what's in it, but it's my thesis and it matters to me :)

Edit: you can see some of the results here btw:

https://github.com/stassa/louise

replies(1): >>41888487 #
5. upghost ◴[19 Oct 24 15:48 UTC] No.41888487{4}[source]▶
Amazing!! There was a chapter on this in my Prolog AI [1] book but I quickly realized it was a superset of the difficulty of high level Prolog programming, high level symbolic AI, and complex composition, and I knew it would have to wait until I graduated from white belt Prolog, but I am incredibly excited to see a real project using this technique!! I remember when I went for my masters degree in AI/ML, the industry was just moving away from decision trees into NNs, but in general we were already well into the shift to subsymbolic "function approximator" style "AI". In fact, the term "AI" was generally poo-pooed in favor of the more technical term "machine learning". I was heart broken, because I was really disappointed to see that AI was less about beautiful programs and more about cleaning and cramming data into a network.

So in a lot if ways seeing this restores some faith in humanity, great work and thanks for giving me a chance to look at it!

With all the hot news in Prolog these days I'd think you should submit this! But also I hate submitting any of my own work and prefer to live in the comment section so I'd understand if you feel the same way.

[1] https://books.google.com/books/about/Prolog_Programming_for_...

replies(1): >>41891610 #
6. YeGoblynQueenne ◴[19 Oct 24 23:24 UTC] No.41891610{5}[source]▶
Oh, I didn't remember that Bratko had a chapter on ILP. I've met him in a couple of ILP conferences so I knew he's published work in ILP though [1]. The techniques described in the book are quite a bit older and, to be honest, they were rather limited, in particular with respect to learning recursion. There's a New Wave of ILP nowadays however and a flourishing of new approaches that followed from MIL, which kinda threw open the gates.

There's a recent synopsis of the latest advances in ILP here:

https://arxiv.org/pdf/2102.10556

>> With all the hot news in Prolog these days I'd think you should submit this!

You mean to HN? I guess I could. I tend to think HN will not find it particularly interesting. Go ahead and submit it yourself though if you feel like it :)

_________________

[1] One of those times Bratko told me that I'm good with Prolog. I replied that I'm going to be saying he said that to everyone who will listen for the rest of my life XD

replies(1): >>41905953 #
7. upghost ◴[21 Oct 24 16:47 UTC] No.41905953{6}[source]▶
YAY!! Thank you! I feel like there is "so much Prolog" that is not really widely known about, it's not like it's burning up the blogosphere unfortunately. I had no idea the ILP methods in the Bratko book were already so dated. Are these Prolog specific conferences you go to or general ML ones...? Where do all the cool kids hang out??