Awesome!
replies(1):
Where I am, it's perfectly legal.
Before cell service was as widespread as it is today, there were programs that would scrape web pages into ePUBs so you could read them later on your Palm Pilot. I used it every day during my commute. And the best part was that they ended. No mind-numbing infinite scroll.
When I switched to a "smart" phone (SonyEricsson m600c), I really missed it.
I spend enough time at a computer than I shouldn't really need a smartphone outside of 'I need to message ___' or 'I need to go ___'
I once made a simple version of this concept that saves an epub file on the server‘s file system, which is then synced to my e-book reader:
https://github.com/solarkraft/webpub
The main ingredient is Postlight Parser, which gives a simplified „document“ view for a website.
On the one hand, they'd be adding a massive amount of free content to a platform where they make money because people pay to consume content.
On the other hand, it might actually increase sales simply because I'd spend more time using it, which would presumably result in more book purchases too.
(Also Kindle store is already full of $0 public domain stuff, so they already don't seem too bothered by that possibility.)
A web scraper for blogs and mainly web novels etc and ePub parser that persists the data to database along with categories and tags, and a companion PWA for offline reading to track reading progress on various stories and let me keep multiple versions of the same story (web novels and published epub).
https://medium.com/@chrisdesalvo/the-future-that-everyone-fo...
Are they an amazon offer or do third parties take the time to set that up?
If you start selling the resulting files, now that would be a copyright violation. German law has a right to create a "Privatkopie", i.e. a private copy. I guess this is similar to fair use in US law?
https://sr.ht/~lioploum/offpunk/
Instead of Epub, it get catched down into text files (Gopher), Gemini files (Gemini) and HTML+images (Web Pages). You can visit the hier from ~/.cache/offpunk or directly from Offpunk.
With the "tour" function, forget about doomscrolling. You'll read all the articles in text mode sequentially until you finish down.
It sort of works ie some stories just work others just get the first page.
(dartharva's comment was the only thing here when I first looked from the front page)
This is for ad hoc generation of EPub from websites that don't have scrape well using traditional scrapers (think standard request based command line scripts or some other chrome extensions that scrape based on open tabs/window) for some reasons:
1. Usually command line scrapers and other extensions have predefined sites they work for, this one's outside of those sites
2. Or they requires nontrivial configuration and/or code
3. Some sites use javascript to dynamically generates/retrieve the text, in which case you need the browser to run the JS - This was the biggest gap for me.
4. This one runs in the browser, so maybe less likely to be detected and blocked
I also don't intend this scraper to be robust or used in a repeated fashion as a background scheduled job, that's why there's a UI for selecting key elements for scraping. It's meant to be more generalized so that you don't have to have a configuration for a site to still be able to scrape it relatively easily with just some mouse clicks.
If the site you're scraping is already handled by the other programs/extensions, then this wouldn't perform better since the other ones are specifically configured for those sites. Otherwise, this extension gives you the tool to scrape something once or twice without spending too much time coding/configuring.
I don't find myself sticking to the same site a lot, so wrote this.