←back to thread

208 points henrijn | 1 comments | | HN request time: 0s | source
Show context
ksenzee ◴[] No.42160131[source]
The infrastructure has been rock-solid. I’ve never seen a service grow this fast without any noticeable outages. The architecture and execution are obviously informed by serious experience at Twitter, but it’s also clear that management is giving them everything they need to do the job right.
replies(4): >>42160170 #>>42160324 #>>42160542 #>>42163517 #
nicce ◴[] No.42160324[source]
> The infrastructure has been rock-solid. I’ve never seen a service grow this fast without any noticeable outages. The architecture and execution are obviously informed by serious experience at Twitter, but it’s also clear that management is giving them everything they need to do the job right.

The world has changed quite bit. If you have deep pockets and you can use AWS etc., it isn't a major problem anymore. However, if they indeed run it on their own data centers, that is impressive.

replies(4): >>42160426 #>>42160457 #>>42160493 #>>42161085 #
crazygringo ◴[] No.42160493[source]
> it isn't a major problem anymore

This is not true at all. The hard part isn't cloud vs. on-premise, it's the architecture.

Most sites can either put all their data in a single massive database, or else have an obvious dimension to shard by (e.g. by user ID if users mostly interact with their own data).

But sites where the data is many-to-many and there's a firehose of writes, of which Twitter is a prime example, are a nightmare to scale while remaining performant and reliable. Every single user gets an updated live feed of tweets drawn from every other user -- handling millions of users simultaneously is not easy.

replies(2): >>42160884 #>>42181045 #
nicce ◴[] No.42160884{3}[source]
> But sites where the data is many-to-many and there's a firehose of writes, of which Twitter is a prime example, are a nightmare to scale while remaining performant and reliable. Every single user gets an updated live feed of tweets drawn from every other user -- handling millions of users simultaneously is not easy.

It is definitely not easy. But the core problem of this has been discussed since the release of Facebook. There are very obvious architectures which you can follow and then fix the bottlenecks with money. The cost is still the most relevant problem, which I wanted to say. The current cloud enables much higher optimization threshold and error margin.

replies(1): >>42161231 #
1. lazystar ◴[] No.42161231{4}[source]
bingo. this problem is known and will soon be offered as a fully managed service - 1 click to have your own private social network.