←back to thread

214 points optimalsolver | 1 comments | | HN request time: 0s | source
Show context
egberts1 ◴[] No.45771044[source]
It's simple. Don't ingest more than 40KB at a time into its LLM's RAG pipe and its hallucination goes way, way down.

Preferably like not at the start and best not to do more than 40KB at a time at all.

That's how I learned how to deal with nftables' 120KB parser_bison.y file by breaking them up into clean sections.

All of a sudden, a fully-deterministic LL(1) full semantic pathway of nftables' CLI syntax appears before my very eye (and spent hours validating it): 100% and test generators now can permutate crazy test cases with relative ease.

Cue in Joe Walsh's "Life's Been Good To Me".

replies(1): >>45771097 #
bob_theslob646 ◴[] No.45771097[source]
Why 40kb?
replies(2): >>45772483 #>>45773126 #
1. egberts1 ◴[] No.45773126[source]
Cheap public offering of their expensive data center is that sweet spot and cutoff at 40KB.