←back to thread

420 points rvz | 1 comments | | HN request time: 0.001s | source
Show context
pfraze ◴[] No.41412758[source]
Copying over my latest backend status update; figure folks would find it interesting

Servers are holding up so far! Fortunately we were overprovisioned. If we hit 4mm new signups then things should get interesting. We did have some degradations (user handles entering an invalid state, event-stream crashed a couple times, algo crashed a couple times, image servers hit bad latencies) but we managed to avoid a full outage.

We use an event-sourcing model which is: K/V database for primary storage (actually sqlite), into a golang event stream, then into scylladb for computed views. Various separate services for search, algorithms, and images. Hybrid on-prem & cloud. There are ~20 of the k/v servers, 1 event-stream, 2 scylla clusters (I believe).

The event-stream crash would cause the application to stop making progress on ingesting events, but we still got the writes, so you'd see eg likes failing to increment the counter but then magically taking effect 60 seconds later. Since the scylla cluster and the KV stores stayed online, we avoided a full outage.

replies(9): >>41412984 #>>41413343 #>>41413506 #>>41413569 #>>41415242 #>>41415812 #>>41416225 #>>41417516 #>>41417547 #
pcwalton ◴[] No.41413569[source]
It's frustrating that anything related to X/Twitter is such a predictably-partisan tinderbox because this is really interesting technical information. Thank you for sharing it!
replies(1): >>41414443 #
kstenerud ◴[] No.41414443[source]
It's partisan/political because Musk is partisan/political. And it's not just Musk.

We've been living in a fantasy land of "no political affiliation" in the tech world for decades, and now that the age of the hyper-rich has come once again, they are realizing the benefits of using the power they wield to shape the worlds they live in.

So now in the early stages of this century's great fight, we'll see our beloved tech giants join the political fray in full force, dragging their follower armies along for the ride.

And it works, too. Just look at the comments here.

replies(9): >>41414549 #>>41414566 #>>41414571 #>>41414596 #>>41414987 #>>41415229 #>>41415701 #>>41416661 #>>41417566 #
nox101 ◴[] No.41414596[source]
so strange for you to blame this on Musk. Twitter was already super partisan long before he took it over
replies(3): >>41414616 #>>41414672 #>>41415420 #
JumpCrisscross ◴[] No.41414616[source]
> Twitter was already super partisan long before he took it over

Sure. But Elon changed teams. He used to be bipartisan. But he chose a champion in the aftermath of Covid and--by the looks of it--he's chosen a bad one.

(In an alternate universe where Musk stuck to what he's good at, I could see the entire Artemis programme being delegated to SpaceX and a bipartisan adoption of Tesla as America's EV standard bearer. Instead, there is real political capital in creating a rival to SpaceX. And Tesla is going to have to constantly be on the defence against cheap Chinese imports from the Democrats and establishment Republicans.)

replies(4): >>41414695 #>>41414984 #>>41415488 #>>41415814 #
crossroadsguy ◴[] No.41414984[source]
> Musk stuck to what he's good at

Right. And that is?

replies(2): >>41415012 #>>41415414 #
blueredmodern[dead post] ◴[] No.41415414[source]
[flagged]
1. jajko ◴[] No.41416575[source]
He was able to find or buy people and companies smart and capable enough to deliver this for him, and he was often, but not always smart enough to listen to them (ie tesla autopilot fiasco - example of an ego playing with him). He was so successful with spacex because nobody really dared/bothered to enter that space and existing companies had 0 pressure to deliver better but that's more of government fail. He personally didn't come up with any of those impressive feats like electric cars of vertical rocket landings, thats a ridiculous proposition when all he is is a competent manager.

Now he is slowly rolling back some of those hard won achievements, and competition catching up and/or overtaking in many others.