Most active commenters
  • seestem(4)
  • (3)

←back to thread

The Internet Archive is back online

(arstechnica.com)
339 points Wingy | 24 comments | 16 Oct 24 11:09 UTC | HN request time: 0.662s | source | bottom
Show context
seestem ◴[16 Oct 24 12:10 UTC] No.41858155[source]▶
It would be better if the Internet Archive was decentralized without a central point of failure, maybe run on something like bittorent.
replies(8): >>41858170 #>>41858243 #>>41858272 #>>41858279 #>>41858590 #>>41859340 #>>41860089 #>>41861043 #
1. Cheer2171 ◴[16 Oct 24 12:26 UTC] No.41858279[source]▶
You say this as if it is an original idea. Of course the IA is working on this and have been for over 6 years. There already is a DWeb version. They have been advancing DWeb infrastructure. The IA hosts all kinds of DWeb developer events.

But it is over 50 petabytes and the IA gets a huge amount of traffic through the regular web that they need to serve quickly and efficiently to their users.

Guess what has happened over 6 years of decentralization of 50 TB? People only seed what they want or care about and there aren't enough seeders to host. They set all this up and nobody volunteers. You're a DWeb advocate and you haven't been seeding. That's a recipe for disaster if they rely on the goodness of volunteer seeders. The IA's mission is broader. DWeb will ever only compliment the IAs mission.

https://blog.archive.org/2021/02/18/behind-the-scenes-of-the...

https://www.bleepingcomputer.com/news/technology/archiveorg-...

replies(8): >>41858315 #>>41858402 #>>41858496 #>>41858621 #>>41858851 #>>41859300 #>>41860711 #>>41861723 #
2. seestem ◴[16 Oct 24 12:30 UTC] No.41858315[source]▶
It is not my original idea, but it is an obvious idea to anyone who knows how the internet works, I just added it to get the discussion going.
replies(1): >>41858355 #
3. Cheer2171 ◴[16 Oct 24 12:35 UTC] No.41858355[source]▶
It already exists. IA has had a DWeb for 6 years. Nobody seeds it.
replies(2): >>41858418 #>>41861742 #
4. ◴[16 Oct 24 12:41 UTC] No.41858402[source]▶
5. seestem ◴[16 Oct 24 12:43 UTC] No.41858418{3}[source]▶
Incentivising seeding is hard. Maybe cryptocurrencies can be useful here, but I understand not everyone likes them especially here on HN. In retrospect the ideal setup would have been if archiving was included into the core HTTP protocol.
replies(4): >>41858442 #>>41858806 #>>41858994 #>>41859181 #
6. lukas099 ◴[16 Oct 24 12:51 UTC] No.41858496[source]▶
If I help seed this DWeb and it turns out it has some copyrighted materials in it, will I be potentially held liable?
replies(2): >>41859297 #>>41861479 #
7. ksymph ◴[16 Oct 24 13:04 UTC] No.41858621[source]▶
How does one contribute? In the article you linked:

> there is no information on how users can get involved in the decentralized version of Archive.org and who the peers are that are distributing the content.

The other link doesn't mention how people could help host data either. If there is a way, then it seems like more of a marketing issue if those willing are unaware or unable to figure out how. I can't find any actionable steps on how to contribute.

edit - it seems the dweb version was a frontend for archive.org testing serving IA content over alternative protocols. It was never finished or expanded on unfortunately. Links to it are dead but here's the github repo https://github.com/internetarchive/dweb-archive

8. 6footgeek ◴[16 Oct 24 13:04 UTC] No.41858627{5}[source]▶
Actually I think their wild ideas are contributing to this discussion.. I think your apparent hurt at this suggestion is a bit silly tbh.

Hear me out. I hadn't heard of the prior work you are raging about. But I am interested in decentralised tech - so, by this person mentioning their idea it got me interested to read further. Then you entered and went a bit madlad on them.. I'm not sure why when you could have just said "good question, here is everything we know about the effort to do exactly this" and then maybe that hacker would have been inspired to go read and maybe contribute. Just my 2 cents ;-)

9. ◴[16 Oct 24 13:15 UTC] No.41858718{5}[source]▶
10. Cthulhu_ ◴[16 Oct 24 13:26 UTC] No.41858806{4}[source]▶
Cryptocurrencies implies that people would pay / get paid for it... just pay the directly IA then, or your own servers. Cryptocurrencies imply someone's skimming off a lot for their own pockets.
replies(1): >>41858904 #
11. Mistletoe ◴[16 Oct 24 13:30 UTC] No.41858851[source]▶
https://www.reddit.com/r/DataHoarder/comments/h02jl4/lets_sa...

I’ve always been fascinated by this post.

12. seestem ◴[16 Oct 24 13:36 UTC] No.41858904{5}[source]▶
I meant more like in bitcoin how the miners get paid for mining, or how validators are rewarded in proof of stake blockchains. This techniques can be used to incentivised seeders.
13. usr1106 ◴[16 Oct 24 13:46 UTC] No.41858994{4}[source]▶
Maybe someone can invent a proof of seeding protocol? So that would bring some good to the public instead of just burning energy. Don't ask me how it would work...
replies(1): >>41859531 #
14. ◴[16 Oct 24 13:46 UTC] No.41858996{5}[source]▶
15. greenie_beans ◴[16 Oct 24 14:06 UTC] No.41859181{4}[source]▶
i'm doubtful that whatever crypto incentive that is offered will make up the cost for me to DIY this in my home. which is why crypto miners scale, making crypto a centralized system. i don't care what the latest white paper says, it is still controlled by few people and not decentralized. in the same way that the ussr replicated the centralization of american capitalism, crypto replicates the centralization of trust while marketed as something else.
16. diggan ◴[16 Oct 24 14:19 UTC] No.41859297[source]▶
You're always responsible for what you, yourself and your computer does. There is a chance EFF/some other organization could help you out in case you end up in court, but that's a maybe, not a guarantee.
replies(1): >>41859525 #
17. tomrod ◴[16 Oct 24 14:19 UTC] No.41859300[source]▶
You're describing a network effects problem, specifically a collaborative game failure. Need some mechanism designers and big tech cos to jump in, stat!
18. nikisweeting ◴[16 Oct 24 14:35 UTC] No.41859525{3}[source]▶
Harder to make this argument with encrypted distributed filesystems. If I'm storing a single chunk of an encrypted blob on Filecoin, am I responsible for the entire file even if I don't know what's in it, and I'm only storing a single fragment?
replies(2): >>41860482 #>>41861639 #
19. nikisweeting ◴[16 Oct 24 14:36 UTC] No.41859531{5}[source]▶
Storj, Filecoin, etc. fill this gap but it's still really hard to earn enough to justify the effort at small scales.
20. CaptArmchair ◴[16 Oct 24 15:48 UTC] No.41860482{4}[source]▶
This depends on the jurisdiction you're in. I.e. Europe's GDPR argues that you need consent to keep someone's personal data. Encryption doesn't equate anonymization, so there's a potential liabity.
21. tonetegeatinst ◴[16 Oct 24 16:03 UTC] No.41860711[source]▶
Can confirm that issue about people only seeding what they are interested in.

I found a dataset I wanted to hoard but the authors website was gone. A dataset site had a torrent and I said great I'll just torrent and seed that and help keep the thing alive, turns out I can't find a single seeder for the torrent.

22. akudha ◴[16 Oct 24 17:20 UTC] No.41861479[source]▶
This.

Until a clear, precise answer to this question is available, it is unreasonable to expect individuals to take risks and seed.

It is one thing if an organization like IA gets in trouble with the law. They have money, lawyers, name recognition and are big enough to at least fight a lawsuit, even if they lose. Who is going to help an individual if he/she gets in trouble with the law, unknowingly? Am I expected to read through tons of complex copyright law and interpret it, just so I can seed a handful of items? No thanks.

23. creer ◴[16 Oct 24 17:45 UTC] No.41861723[source]▶
It seems to me the various efforts are dead or stalled. Anything in actual current development or production? IPFS was supposed to go in that direction and still exists, sure, but not to provide IA duplication (that is advertised.)
24. creer ◴[16 Oct 24 17:47 UTC] No.41861742{3}[source]▶
If nobody seeds it (or continue development) then it's dead. Inspiration and perhaps code for the next effort sure. But not "exist" that makes a difference.