Most active commenters
  • eviks(4)
  • nemothekid(3)

←back to thread

492 points storf45 | 19 comments | | HN request time: 0.663s | source | bottom
Show context
shermantanktop ◴[] No.42160502[source]
Every time a big company screws up, there are two highly informed sets of people who are guaranteed to be lurking, but rarely post, in a thread like this:

1) those directly involved with the incident, or employees of the same company. They have too much to lose by circumventing the PR machine.

2) people at similar companies who operate similar systems with similar scale and risks. Those people know how hard this is and aren’t likely to publicly flog someone doing their same job based on uninformed speculation. They know their own systems are Byzantine and don’t look like what random onlookers think it would look like.

So that leaves the rest, who offer insights based on how stuff works at a small scale, or better yet, pronouncements rooted in “first principles.”

replies(15): >>42160568 #>>42160576 #>>42160579 #>>42160888 #>>42160913 #>>42161148 #>>42161164 #>>42161399 #>>42161529 #>>42161703 #>>42161724 #>>42161889 #>>42165352 #>>42166894 #>>42167814 #
1. survirtual ◴[] No.42160913[source]
For an event like this, there already exists an architecture that can handle boundless scale: torrents.

If you code it to utilize high-bandwidth users upload, the service becomes more available as more users are watching -- not less available.

It becomes less expensive with scale, more available, more stable.

The be more specific, if you encode the video in blocks with each new block hash being broadcast across the network, just managing the overhead of the block order, it should be pretty easy to stream video with boundless scale using a DHT.

Could even give high-bandwidth users a credit based upon how much bandwidth they share.

With a network like what Netflix already has, the seed-boxes would guarantee stability. There would be very little delay for realtime streams, I'd imagine 5 seconds top. This sort of architecture would handle planet-scale streams for breakfast on top of the already existing mechanism.

But then again, I don't get paid $500k+ at a large corp to serve planet scale content, so what do I know.

replies(5): >>42161009 #>>42161094 #>>42161365 #>>42161475 #>>42162174 #
2. nemothekid ◴[] No.42161009[source]
Torrents are awful for live events.

1. Everyone only cares about the most recent "block". By the time a "user" has fully downloaded a block from Netflix's seedbox, the block is stale, so why would any other user choose to download from a peer rather from netflix directly?

2. If all the users would prefer to download from netflix directly rather than a p2p user, then you already have a somewhat centralized solution, and you gain nothing from torrents.

replies(2): >>42161197 #>>42161766 #
3. Edman274 ◴[] No.42161094[source]
The protocol for a torrent is that random parts of a file get seeded to random people requesting a file, and that the clients which act as seeds are able to store arbitrary amounts of data to then forward to other clients in the swarm. Do the properties about scaling still hold when it's a bunch of people all requesting real time data which has to be in-order? Do the distributed Rokus, Apple TVs, Fire TVs and other smart TVs all have the headroom in compute and storage to be able to simultaneously decode video and keep old video data in RAM and manage network connections with upload to other TVs in their swarm - and will uploading data to other TVs in the swarm not negatively impact their own download speeds?
replies(2): >>42161166 #>>42163739 #
4. alex-mohr ◴[] No.42161166[source]
Yes, the properties about scaling do hold even with near-real-time streams. [1]

The problems with using it as part of a distributed service have more to do with asymmetric connections: using all of the limited upload bandwidth causes downloads to slow. Along with firewalls.

But the biggest issue: privacy. If I'm part of the swarm, maybe that means I'm watching it?

[1]: Chainsaw: P2P streaming without trees, https://link.springer.com/chapter/10.1007/11558989_12

5. alex-mohr ◴[] No.42161197[source]
If Netflix were working correctly and could handle the load, you'd absolutely be correct.

But it does seem the capacity of a hybrid system of Netflix servers plus P2P would be strictly greater than either alone? It's not an XOR.

And note that in this case of "live" streaming, it still has a few seconds of buffer, which gives a bandwidth-delay product of a few MB. That's plenty to have non-stale blocks and do torrent-style sharing.

replies(1): >>42162094 #
6. transcriptase ◴[] No.42161365[source]
I don’t pay my ISP each month to be part of a streaming sites infrastructure. I pay the streaming site each month to use theirs.
replies(2): >>42161775 #>>42169555 #
7. miki123211 ◴[] No.42161475[source]
Then, instead of people complaining about buffering issues, you'd get people complaining about how the greedy capitalists at Netflix made poor Joe Shmoe use all of his data cap, because they made him upload lots of data to other users and couldn't be bothered to do it themselves.
8. eviks ◴[] No.42161766[source]
1. Because Netflix is at capacity? Or because the peer is closer and faster than the original?
replies(1): >>42162084 #
9. eviks ◴[] No.42161775[source]
And you'll pay less if you become a part
replies(1): >>42161941 #
10. transcriptase ◴[] No.42161941{3}[source]
Sure. If there’s anything publicly traded companies are known for, it’s passing savings onto their customers instead of their shareholders.
replies(1): >>42162140 #
11. nemothekid ◴[] No.42162084{3}[source]
If Netflix is at capacity and you have to wait for a peer, then you have simply reinvented the buffering problem. In other words

1. I exclusively download from a peer and my stream is measurably behind

2. I switch to a peer when Netflix is at capacity and then I have to wait for the peer to download from Netflix, and then for me to download from the peer. This will cause the same buffering issue that Netflix is currently being lambasted for.

This solution doesn’t solve the problem Netflix has

replies(1): >>42162129 #
12. nemothekid ◴[] No.42162094{3}[source]
If switching to a peer causes increased buffering (which it will, because you still have to wait for the peer to download from Netflix) then you will still have the original problem Netflix is suffering from.

If the solution to users complaining about buffering is to build a system with more inherent buffering then you are back at square one.

I think it’s might be helpful to look at netlfix’s current system as already a distributed video delivery system in which they control the best seeds. Adding more seeds may help, but if Netflix is underprovisioned from the start you will have users who cannot access the streams

13. eviks ◴[] No.42162129{4}[source]
"Buffering problem" can have very different QOL manifestations, so :

1. You still get a better viewing experience without interruptions. Besides, your "measurably behind" can be an imperceptible fraction of a second?

2. Similar thing - shorter queues - the switch can happen faster due to the extra capacity

So yes, it does solve the practical problem, though not the theoretical one

14. eviks ◴[] No.42162140{4}[source]
The knowledge about that universal practice can be easily acquired.
15. kmeisthax ◴[] No.42162174[source]
Yes, and then some idiot with an axe to grind against Logan Paul starts DDoSing people in the Netflix swarm, kicking them out of the livestream. This is always a problem because torrents, by design, are privacy-hostile. That's how the MAFIAA[1] figured out you were torrenting movies in 2004 and how they sent your ISP a takedown notice.

Hell, in the US, this setup might actually be illegal because of the VPPA[0]. The only reason why it's not illegal for the MAFIAA to catch you torrenting is because of a fun legal principle where criminals are not allowed to avail themselves of the law to protect their crimes. (i.e. you can't sue over a drug deal gone wrong)

[0] Video Privacy Protection Act, a privacy law passed which makes it illegal to ask video providers for a list of who watched what, specifically because a reporter went on a fishing expedition with video data.

[1] Music and Film Industry Association of America, a hypothetical merger of the MPAA and RIAA from a 2000s era satire article

replies(1): >>42166478 #
16. survirtual ◴[] No.42163739[source]
Use your imagination for just a moment.

The torrent is an example of the system I am describing, not the same system. Torrents cannot work for live streams because the entire content is not hashable yet, so already you have to rethink how it's done. I am talking about adding a p2p layer on top of the existing streaming protocol.

The current streaming model would prioritize broadcasting to high-bandwidth users first. There should be millions of those in a world-scale stream.

Even a fraction of these millions would be enough to reduce Netflix's streaming costs by an order of magnitude. But maybe Netflix isn't interested in saving billions?

With more viewers, the availability of content increases, which reduces load on the centralized servers. This is the property of the system I am talking about, so think backwards from that.

With a livestream, you want the youngest block to take priority. You would use the DHT to manage clients and to manage stale blocks for users catching up.

The youngest block would be broadcast on the p2p network and anyone who is "live" would be prioritizing access to that block.

Torrent clients as they are now handle this case, in reverse; they can prioritize blocks closer the current timestamp to created an uninterrupted stream.

The system I am talking about would likely function at any scale, which is an improvement from Netflix's system, which we know will fail -- because it did.

17. w0mbat ◴[] No.42166478[source]
Tyson was fighting Jake Paul, not Logan Paul. That’s Jake’s brother.
18. stainforth ◴[] No.42169555[source]
If you use Comcast's modem/wifi router, you are part of their service infrastructure. Xfinity WiFi Home Hotspot
replies(1): >>42187664 #
19. jhowison ◴[] No.42187664{3}[source]
Yes, it's on by default, but you can turn this off if you want to. https://www.xfinity.com/support/articles/disable-xfinity-wif...