The Bluesky firehose viewed in the style of a Windows XP screensaver

Yeah I was looking into the firehose as a potential way to source to discover new domains for my search engine. Even though it didn't pan out, I really appreciate how accessible the data is.

Why didn't it work out?

I decided agains it because it had an incredibly bad signal to noise ratio. Almost all links I saw were either to big websites like newspapers, patreon, onlyfans; or behind url shorteners.

Dunno, I may explore it further down the line, but for now the juice didn't seem worth the squeeze.

I created a website like 10 years ago called birdmine that indexed every link you or one of your followers shared on Twitter, in a Solr search engine so you could search stuff that had been curated to an extent. It was pretty cool, I think I’m the only person that ever used it though.

BSky could use a better search, or a better curation of feeds. I'm certain there is more room in the discovery play.

I might have asked this before, but did you look at adding this type of ingestion for ActivityPub?

Briefly, but I've come to learn there's a contingent of aggressively search-engine hostile people that has made a home on the fediverse. The federated nature of it makes it somewhat tricky to untangle the search engine friendly people from the hostile.

I don't need the inevitable DDOS:es and death threats you get when upsetting a clique of mentally ill people online.

You're right, I forgot about the backlash that this type of efforts got from people that don't understand the technology and just make assumptions about element visibility in the network, though maybe calling them mentally ill is a bit much.

The sorts threatening physical violence almost certainly are. But it's a minority to be sure.