(www.boundaryml.com)

169 points constantinum | 1 comments | 18 Jun 24 04:01 UTC | HN request time: 0.207s | source

Show context

zora_goron ◴[18 Jun 24 08:04 UTC] No.40715234[source]▶

The article mentions,

>> “you've tried response_format: "json" and function calling and been disappointed by the results”

Can anyone share any examples of disappointments or issues with these techniques? Overall I’ve been pretty happy with JSON mode via OpenAI API so I’m curious to hear about any drawbacks with it.

replies(2): >>40715280 #>>40715292 #

1. hellovai ◴[18 Jun 24 08:16 UTC] No.40715292[source]▶

>>40715234 #

The main drawback is really when you attempt to do more advanced prompting techniques like chain-of-thought or reasoning.

forcing those parts to be json, can be hard and unnecessarily constrain the model. e.g. https://www.promptfiddle.com/Chain-of-Thought-KcSBh

try pressing run tests and you'll see what i mean! this method or doing chain of thought works a bit better

↑

Every Way to Get Structured Output from LLMs