It would be better if the Internet Archive was decentralized without a central point of failure, maybe run on something like bittorent.
replies(8):
Bittorrent works well for popular things but fails for marginal content (unless some really dedicated individuals step in.)
What the internet archive provides is a way to have access to many many resources which you didn't know you needed in advance.
At that time our son was using it for games. He goes away to college and came home for the first school break. I get a phone call from our internet provider asking if our son was home. I was so shocked and handed the phone to our son.
Apparently at that time bittorrent was optimizing for the most efficient path to a host. Since we had relatively good connection, the mighty weight of the internet was funnelling through our tiny internet provider to our son's computer. The provider (without our knowing it) had made a deal with our son that he would only turn on bittorrent between midnight and 6 AM. I doubt other providers would be so generous.
I have been sceptical of bittorrent since that day.
But it is over 50 petabytes and the IA gets a huge amount of traffic through the regular web that they need to serve quickly and efficiently to their users.
Guess what has happened over 6 years of decentralization of 50 TB? People only seed what they want or care about and there aren't enough seeders to host. They set all this up and nobody volunteers. You're a DWeb advocate and you haven't been seeding. That's a recipe for disaster if they rely on the goodness of volunteer seeders. The IA's mission is broader. DWeb will ever only compliment the IAs mission.
https://blog.archive.org/2021/02/18/behind-the-scenes-of-the...
https://www.bleepingcomputer.com/news/technology/archiveorg-...
Unless I'm missing something, an archive is not something small or something that's just as good when part of it is missing.
> there is no information on how users can get involved in the decentralized version of Archive.org and who the peers are that are distributing the content.
The other link doesn't mention how people could help host data either. If there is a way, then it seems like more of a marketing issue if those willing are unaware or unable to figure out how. I can't find any actionable steps on how to contribute.
edit - it seems the dweb version was a frontend for archive.org testing serving IA content over alternative protocols. It was never finished or expanded on unfortunately. Links to it are dead but here's the github repo https://github.com/internetarchive/dweb-archive
Hear me out. I hadn't heard of the prior work you are raging about. But I am interested in decentralised tech - so, by this person mentioning their idea it got me interested to read further. Then you entered and went a bit madlad on them.. I'm not sure why when you could have just said "good question, here is everything we know about the effort to do exactly this" and then maybe that hacker would have been inspired to go read and maybe contribute. Just my 2 cents ;-)
I’ve always been fascinated by this post.
I think Archive.org should still exist too (and ArchiveBox donates + submits URLs to Archive.org too), but having a self-hosted option where you can archive personal stuff that requires a login, and do P2P sharing with with fine grained permissions is a gap that should be filled.
Aiming to archive the entire internet is Archive.org's goal, aiming to archive the part of the internet YOU care about is our goal.
[I know that some percentage between 95 and 100 of crypto projects are a scam. I personally believe this one isn't, after much diligent reading. Whether it gets released or does what it claims it will do is another question, but please do spare me the kneejerk anti-crypto reactions, if you can. Just because they're almost all money-making scams, doesn't mean they're all money-making scams.]
I found a dataset I wanted to hoard but the authors website was gone. A dataset site had a torrent and I said great I'll just torrent and seed that and help keep the thing alive, turns out I can't find a single seeder for the torrent.
Until a clear, precise answer to this question is available, it is unreasonable to expect individuals to take risks and seed.
It is one thing if an organization like IA gets in trouble with the law. They have money, lawyers, name recognition and are big enough to at least fight a lawsuit, even if they lose. Who is going to help an individual if he/she gets in trouble with the law, unknowingly? Am I expected to read through tons of complex copyright law and interpret it, just so I can seed a handful of items? No thanks.