Every Way to Get Structured Output from LLMs

(www.boundaryml.com)

169 points constantinum | 5 comments | 18 Jun 24 04:01 UTC | HN request time: 0s | source

Show context

resiros ◴[18 Jun 24 12:55 UTC] No.40717267[source]▶

I expected to read about the methods used by the libraries to get the structured output and not a comparison of the language compatibility for each.

Fortunately the same author have a blog post (https://www.boundaryml.com/blog/type-definition-prompting-ba...) explaining how their approach works and how it compares to instructor (https://github.com/jxnl/instructor).

Basically these libraries provide two things: 1. A way to prompt the LLM 2. A way to get a valid JSON

For 1. instructor does it through the json schema definition, BAML's innovation is that they use a simplified lossless schema definition that uses less tokens.

For 2. instructor does it through reprompting until they receive a valid JSON. BAML's innovation is a fuzzy parser able to to parse non-perfect JSON.

Personally I think that there is no need to all these abstractions to get structured outputs from LLMs. A simple .to_prompt() function that takes a pydantic and translate it into some prompt block you can add to your prompt and a retry is sufficient to get the same results.

replies(1): >>40717685 #

1. Jayakumark ◴[18 Jun 24 13:33 UTC] No.40717685[source]▶

>>40717267 #

Will you be able to share an example code or gist ?

replies(2): >>40719010 #>>40720535 #

2. vladsanchez ◴[18 Jun 24 15:33 UTC] No.40719010[source]▶

>>40717685 (TP) #

https://agenta.ai/

replies(1): >>40720463 #

3. resiros ◴[18 Jun 24 18:01 UTC] No.40720463[source]▶

>>40719010 #

Thanks for sharing the link, but no agenta is not a library that can help with getting structured outputs from LLMs (at least not in the way discussed in the parent comment). It's a prompt management, evaluation and observability platform for LLM apps.

replies(1): >>40805638 #

4. resiros ◴[18 Jun 24 18:10 UTC] No.40720535[source]▶

>>40717685 (TP) #

If you look into the instructor code(https://github.com/jxnl/instructor/blob/06a49e7824729b8df1f7...). Here is the core code snippet they use:

            message = dedent(
                f"""
                As a genius expert, your task is to understand the content and provide
                the parsed objects in json that match the following json_schema:\n

                {json.dumps(response_model.model_json_schema(), indent=2)}

                Make sure to return an instance of the JSON, not the schema itself
                """
            )

Then depending on the mode, either they add another message `Return the correct JSON response within a ```json codeblock. not the JSON_SCHEMA` or they set the response format to json.

5. vladsanchez ◴[26 Jun 24 23:16 UTC] No.40805638{3}[source]▶

>>40720463 #

Just stumbled with https://controlflow.ai/ today. Perhaps it serves to structure outputs as "agentic" workflows in pursuit of LLM autonomy.

Let us know your opinion.

↑