←back to thread

Google Titans architecture, helping AI have long-term memory

(research.google)
584 points Alifatisk | 1 comments | 07 Dec 25 12:23 UTC | HN request time: 0s | source
Show context
kgeist ◴[07 Dec 25 14:57 UTC] No.46182122[source]▶
>The model uses this internal error signal (the gradient) as a mathematical equivalent of saying, "This is unexpected and important!" This allows the Titans architecture to selectively update its long-term memory only with the most novel and context-breaking information

So one can break a model by consistently feeding it with random, highly improbable junk? Everything would be registered as a surprise and get stored, impacting future interactions

replies(6): >>46182150 #>>46182410 #>>46182651 #>>46183200 #>>46183413 #>>46193429 #
1. photochemsyn ◴[07 Dec 25 17:08 UTC] No.46183200[source]▶
This is no different from what happens to humans if they're locked into cult programming situations, they'll start believing and regurgitating all kinds of nonsense if their information stream is tightly curated,

Practically, for use with a codebase development effort, if the model remembers the original design decisions, the discussions about costs and benefits, then can remember all that much later in the process, it's going to start getting really good at thinking about what the next step is, or even to make decisions about when a major refactor is neede, etc.