Google Titans architecture, helping AI have long-term memory

>The model uses this internal error signal (the gradient) as a mathematical equivalent of saying, "This is unexpected and important!" This allows the Titans architecture to selectively update its long-term memory only with the most novel and context-breaking information

So one can break a model by consistently feeding it with random, highly improbable junk? Everything would be registered as a surprise and get stored, impacting future interactions

In what world can you not always break the response of an AI by feeding it a bunch of random junk?

I mean ideally AI would be resilient to junk, don't you think?