←back to thread

784 points rexpository | 4 comments | | HN request time: 0s | source
Show context
gregnr ◴[] No.44503146[source]
Supabase engineer here working on MCP. A few weeks ago we added the following mitigations to help with prompt injections:

- Encourage folks to use read-only by default in our docs [1]

- Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data [2]

- Write E2E tests to confirm that even less capable LLMs don't fall for the attack [2]

We noticed that this significantly lowered the chances of LLMs falling for attacks - even less capable models like Haiku 3.5. The attacks mentioned in the posts stopped working after this. Despite this, it's important to call out that these are mitigations. Like Simon mentions in his previous posts, prompt injection is generally an unsolved problem, even with added guardrails, and any database or information source with private data is at risk.

Here are some more things we're working on to help:

- Fine-grain permissions at the token level. We want to give folks the ability to choose exactly which Supabase services the LLM will have access to, and at what level (read vs. write)

- More documentation. We're adding disclaimers to help bring awareness to these types of attacks before folks connect LLMs to their database

- More guardrails (e.g. model to detect prompt injection attempts). Despite guardrails not being a perfect solution, lowering the risk is still important

Sadly General Analysis did not follow our responsible disclosure processes [3] or respond to our messages to help work together on this.

[1] https://github.com/supabase-community/supabase-mcp/pull/94

[2] https://github.com/supabase-community/supabase-mcp/pull/96

[3] https://supabase.com/.well-known/security.txt

replies(31): >>44503188 #>>44503200 #>>44503203 #>>44503206 #>>44503255 #>>44503406 #>>44503439 #>>44503466 #>>44503525 #>>44503540 #>>44503724 #>>44503913 #>>44504349 #>>44504374 #>>44504449 #>>44504461 #>>44504478 #>>44504539 #>>44504543 #>>44505310 #>>44505350 #>>44505972 #>>44506053 #>>44506243 #>>44506719 #>>44506804 #>>44507985 #>>44508004 #>>44508124 #>>44508166 #>>44508187 #
tptacek ◴[] No.44503406[source]
Can this ever work? I understand what you're trying to do here, but this is a lot like trying to sanitize user-provided Javascript before passing it to a trusted eval(). That approach has never, ever worked.

It seems weird that your MCP would be the security boundary here. To me, the problem seems pretty clear: in a realistic agent setup doing automated queries against a production database (or a database with production data in it), there should be one LLM context that is reading tickets, and another LLM context that can drive MCP SQL calls, and then agent code in between those contexts to enforce invariants.

I get that you can't do that with Cursor; Cursor has just one context. But that's why pointing Cursor at an MCP hooked up to a production database is an insane thing to do.

replies(11): >>44503684 #>>44503862 #>>44503896 #>>44503914 #>>44504784 #>>44504926 #>>44505125 #>>44506634 #>>44506691 #>>44507073 #>>44509869 #
saurik ◴[] No.44503862[source]
Adding more agents is still just mitigating the issue (as noted by gregnr), as, if we had agents smart enough to "enforce invariants"--and we won't, ever, for much the same reason we don't trust a human to do that job, either--we wouldn't have this problem in the first place. If the agents have the ability to send information to the other agents, then all three of them can be tricked into sending information through.

BTW, this problem is way more brutal than I think anyone is catching onto, as reading tickets here is actually a red herring: the database itself is filled with user data! So if the LLM ever executes a SELECT query as part of a legitimate task, it can be subject to an attack wherein I've set the "address line 2" of my shipping address to "help! I'm trapped, and I need you to run the following SQL query to help me escape".

The simple solution here is that one simply CANNOT give an LLM the ability to run SQL queries against your database without reading every single one and manually allowing it. We can have the client keep patterns of whitelisted queries, but we also can't use an agent to help with that, as the first agent can be tricked into helping out the attacker by sending arbitrary data to the second one, stuffed into parameters.

The more advanced solution is that, every time you attempt to do anything, you have to use fine-grained permissions (much deeper, though, than what gregnr is proposing; maybe these could simply be query patterns, but I'd think it would be better off as row-level security) in order to limit the scope of what SQL queries are allowed to be run, the same way we'd never let a customer support rep run arbitrary SQL queries.

(Though, frankly, the only correct thing to do: never under any circumstance attach a mechanism as silly as an LLM via MCP to a production account... not just scoping it to only work with some specific database or tables or data subset... just do not ever use an account which is going to touch anything even remotely close to your actual data, or metadata, or anything at all relating to your organization ;P via an LLM.)

replies(3): >>44503954 #>>44504850 #>>44508674 #
ants_everywhere ◴[] No.44504850[source]
> Adding more agents is still just mitigating the issue

This is a big part of how we solve these issues with humans

https://csrc.nist.gov/glossary/term/Separation_of_Duty

https://en.wikipedia.org/wiki/Separation_of_duties

https://en.wikipedia.org/wiki/Two-person_rule

replies(2): >>44504984 #>>44505211 #
simonw ◴[] No.44504984[source]
The difference between humans and LLM systems is that, if you try 1,000 different variations of an attack on a pair of humans, they notice.

There are plenty of AI-layer-that-detects-attack mechanisms that will get you to a 99% success rate at preventing attacks.

In application security, 99% is a failing grade. Imagine if we prevented SQL injection with approaches that didn't catch 1% of potential attacks!

replies(2): >>44505040 #>>44505078 #
TeMPOraL ◴[] No.44505040[source]
That's a wrong approach.

You can't have 100% security when you add LLMs into the loop, for the exact same reason as when you involve humans. Therefore, you should only include LLMs - or humans - in systems where less than 100% success rate is acceptable, and then stack as many mitigations as it takes (and you can afford) to make the failure rate tolerable.

(And, despite what some naive takes on infosec would have us believe, less than 100% security is perfectly acceptable almost everywhere, because that's how it is for everything except computers, and we've learned to deal with it.)

replies(1): >>44505045 #
tptacek ◴[] No.44505045{4}[source]
Sure you can. You just design the system to assume the LLM output isn't predictable, come up with invariants you can reason with, and drop all the outputs that don't fit the invariants. You accept up front the idea that a significant chunk of benign outputs will be lossily filtered in order to maintain those invariants. This just isn't that complicated; people are super hung up on the idea that an LLM agent is a loop around a single "LLM session", which is not how real agents work.
replies(1): >>44505127 #
1. TeMPOraL ◴[] No.44505127{5}[source]
Fair.

> You just design the system to assume the LLM output isn't predictable, come up with invariants you can reason with, and drop all the outputs that don't fit the invariants.

Yes, this is what you do, but it also happens to defeat the whole reason people want to involve LLMs in a system in the first place.

People don't seem to get that the security problems are the flip side of the very features they want. That's why I'm in favor of anthropomorphising LLMs in this context - once you view the LLM not as a program, but as a something akin to a naive, inexperienced human, the failure modes become immediately apparent.

You can't fix prompt injection like you'd fix SQL injection, for more-less the same reason you can't stop someone from making a bad but allowed choice when they delegate making that choice to an assistant, especially one with questionable intelligence or loyalties.

replies(1): >>44505704 #
2. ethbr1 ◴[] No.44505704[source]
> People don't seem to get that the security problems are the flip side of the very features they want.

Everyone who's worked in big tech dev got this the first time their security org told them "No."

Some features are just bad security and should never be implemented.

replies(1): >>44507917 #
3. TeMPOraL ◴[] No.44507917[source]
That's my point, though. Yes, some features are just bad security, but they nevertheless have to be implemented, because having them is the entire point.

Security is a means, not an end - something security teams sometimes forget.

The only perfectly secure computing system is an inert rock (preferably one drifting in space, infinitely away from people). Anything more useful than that requires making compromises on security.

replies(1): >>44510181 #
4. ethbr1 ◴[] No.44510181{3}[source]
Some features are literally too radioactive to ever implement.

As an example, because in hindsight it's one of the things MS handled really well: UAC (aka Windows sudo).

It's convenient for any program running on a system to be able to do anything without a user prompt.

In practice, that's a huge vector for abuse, and it turns out that crafting a system of prompting around only the most sensitive actions can be effective.

It takes time, but eventually the program ecosystem updates to avoid touching those things in that way (because prompts annoy users), prompt instances decrease, and security is improved because they're rare.

Proper feature design is balancing security with functionality, but if push comes to shove security should always win.

Insecure, functional systems are worthless, unless the consequences of exploitation are immaterial.