We use caching a lot, anything that gets cached can only be written by one service each. The writing services emit cache invalidation messages via SNS that cache users must listen to via SQS, to clear/update their cache.
Alternatively we cache stuff with just a TTL, when immediate cache invalidation is not important.
Where‘s the struggle?
If it doesn’t guarantee delivery, then I believe you will at some point have a client that reads a cached value thinking it’s still valid because the invalidation message got lost in the network.
If you don't understand how and why and when eventual consistency is a problem, you will never understand why cache invalidation is hard.
By the sound of your example, you only handle scenarios where naive approaches to cache invalidation serve your needs, and you don't even have to deal with problems caused by spikes to origin servers. That's perfectly fine.
Others do. They understand the meme. You can too if you invest a fee minutes reading up on the topic.
It's used in DNS, which already was an example here. There is no way to be sure clients see an updated value before end of TTL. As a result, you have to use very conservative TTLs. It's very inefficient.
> anything that gets cached can only be written by one service each
How do you guarantee it's only written by one service each? Sounds like locking across network boundaries, which is not easy.
> The writing services emit cache invalidation messages via SNS that cache users must listen to via SQS
SNS and SQS are both nontrivial services (at least you don't have to build / maintain them I suppose) that require training to use effectively and avoid any possible footguns
I think you're underestimating the complexity in your own solution, and you're probably lucky that some of the harder problems have already been solved for you.
1. Be content with/resilient to the possibility of stale data.
2. Gatekeep all reads and writes (for some subset of the key space) through a single thread.
That's basically it.