←back to thread

Every Way to Get Structured Output from LLMs

(www.boundaryml.com)
169 points constantinum | 1 comments | 18 Jun 24 04:01 UTC | HN request time: 0.194s | source
Show context
zora_goron ◴[18 Jun 24 08:04 UTC] No.40715234[source]▶
The article mentions,

>> “you've tried response_format: "json" and function calling and been disappointed by the results”

Can anyone share any examples of disappointments or issues with these techniques? Overall I’ve been pretty happy with JSON mode via OpenAI API so I’m curious to hear about any drawbacks with it.

replies(2): >>40715280 #>>40715292 #
amake ◴[18 Jun 24 08:14 UTC] No.40715280[source]▶
We specify an output schema (TypeScript syntax) in our system prompt, and OpenAI gets it right most of the time. With some regularity it will give invalid (per the schema) output like

- Return a single object instead of an array of objects

- Return an array of a single object instead of just the object

On the other hand I personally haven't seen it give malformed JSON; the JSON is well-formed but not compliant with the schema we specified.

replies(1): >>40715310 #
1. hellovai ◴[18 Jun 24 08:20 UTC] No.40715310[source]▶
oh thats really interesting, how often do you get errors like that?

fyi, we actually fix those specific errors in our parser :)