←back to thread

809 points rexpository | 1 comments | | HN request time: 0.202s | source
Show context
roadside_picnic ◴[] No.44504625[source]
Maybe I'm getting too old but the core problem here seems to be with `execute_sql` as a tool call!

When I learned database design back in the early 2000s one of the essential concepts was a stored procedure which anticipated this problem back when we weren't entirely sure how much we could trust the application layer (which was increasingly a webpage). The idea, which has long since disappeared (for very good and practical reasons)from modern webdev, was that even if the application layer was entirely compromised you still couldn't directly access data in the data layer.

No need to bring back stored procedure, but only allowing tool calls which themselves are limited in scope seem the most obvious solution. The pattern of "assume the LLM can and will be completely compromised" seems like it would do some good here.

replies(1): >>44504797 #
raspasov ◴[] No.44504797[source]
If the LLM has access to executing only specific stored procedures (I assume modern DBMSs can achieve that granularity, but I haven't checked), then the problem mostly (entirely?) disappears.

It limits the utility of the LLM, as it cannot answer any question one can think of. From one perspective, it's just a glorified REST-like helper for stored procedures. But it should be secure.

replies(2): >>44505188 #>>44508513 #
1. nijave ◴[] No.44508513[source]
You can expose all data with stored procedures.

I'd probably lean towards doing it outside SQL, though (with some other API written in a general purpose programming language)