←back to thread

Betting on DSPy for Systems of LLMs

(blog.isaacmiller.dev)
83 points wavelander | 2 comments | 11 Aug 24 02:11 UTC | HN request time: 0.413s | source
Show context
NeutralCrane ◴[11 Aug 24 05:34 UTC] No.41214178[source]▶
The more I’ve looked at DSPy, the less impressed I am. The design of the project is very confusing with non-sensical, convoluted abstractions. And for all the discussion surrounding it, I’ve yet to see someone actually using for something other than a toy example. I’m not sure I’ve even seen someone prove it can do what it claims to in terms of prompt optimization.

It reminds me very much of Langchain in that it feels like a rushed, unnecessary set of abstractions that add more friction than actual benefit, and ultimately boils down to an attempt to stake a claim as a major framework in the still very young stages of LLMs, as opposed to solving an actual problem.

replies(5): >>41214243 #>>41216015 #>>41216615 #>>41217298 #>>41218357 #
Der_Einzige ◴[11 Aug 24 05:55 UTC] No.41214243[source]▶
Agreed 100%. DSPy along with libraries inspired by it (i.e. https://github.com/zou-group/textgrad) are nothing more than fancy prompt chains under the hood.

These libraries mostly exist as "cope" for the fact that we don't have good fine-tuning (i.e. lora) capabilities for ChatGPT et al, so we try to instead optimize the prompt.

replies(3): >>41214937 #>>41215025 #>>41216691 #
1. isaacbmiller ◴[11 Aug 24 15:04 UTC] No.41216691[source]▶
> These libraries mostly exist as "cope"

> nothing more than fancy prompt chains under the hood

Some approaches using steering vectors, clever ways of fine-tuning, transfer decoding, some tree search sampling-esque approaches, and others all seem very promising.

DSPy is, yes, ultimately a fancy prompt chain. Even once we integrate some of the other approaches, I don't think it becomes a single-lever problem where we can only change one thing(e.g., fine-tune a model) and that solves all of our problems.

It will likely always be a combination of the few most powerful levers to pull.

replies(1): >>41216994 #
2. Der_Einzige ◴[11 Aug 24 15:45 UTC] No.41216994[source]▶
Correct, when I say "ChatGPT et al", I mean closed source paywalled LLMs, open access LLM personalization is an extreme gamechanger. All of what you mentioned is important, and I'm particularly excited about PyReft.

https://github.com/stanfordnlp/pyreft

Anything Christopher Manning touches turns to gold.