Google stops letting sites like Forbes rule search for "Best CBD Gummies"

Hopefully this is a step in the right direction. Google's search results have gotten so bad - seems like even some of the simplest searches are just packed with AI generated and SEO garbage. I don't even want SearchGPT do take over the search market space because I'm almost sure it will still be garbage. Just bring back the google from 5-10 years ago please :(.

You can't go back to the way things were. The world moves forward and changes, and we have to adapt to it.

Web search has always been an extremely messy solution to many problems. Think about the premise: type in anything, and somehow it will read your mind, intuit who you are and what you really wanted, find the exact thing amid the morass of the whole web, and then give it to you?

That's impossible. So it uses tricks to make it seem like it worked. It uses information about you to refine results. It uses curated, human-edited search and result heuristics for the most common or difficult search queries. It uses a giant corups of data, and shows you things that are like what you wanted.

You don't notice that it isn't giving you the best result, because there are so many mediocre-but-acceptable results to look at. And it doesn't have to work perfectly every time, because we can "sift through" results and "refine" our search. Often we are flooded with results that are targeted at us, rather than what we want, because, remember: Google is an advertising company, and the entire Web is now a shopping mall, where either you're being sold-to, or you're just being sold.

You will get results, and they will sort-of seem like what you wanted, so you will just sort of sigh and accept it. Because what other option is there?

There are more intelligent, more accurate, more safe, ways to solve the problems people have, that are not "a search engine". It's time we start implementing them.

Not sure why you’re being downvoted, this is a pointed analysis of why crawl-based search is insufficient for an Internet of our current scale. There is no corporate-curated algorithm that is up to the task, especially when the primary purpose is to profit from advertising.

> You don't notice that it isn't giving you the best result

That's fine. It's always been fine. I don't need Google to read my mind and fulfill my dreams.

The problem isn't that they're not divinely perfect. The problem is that they used to be good enough, and now they're not.

> There are more intelligent, more accurate, more safe, ways to solve the problems people have, that are not "a search engine". It's time we start implementing them.

What solutions are there that fulfill all the use cases of a search engine, while definitively not being a search engine? An AI chatbot that gives me synopses of the same websites that I was searching for does not count.

> Just bring back the google from 5-10 years ago please

What you really want is the internet from 5-10 years ago (really even longer than that), and that's not coming back.

>There is no corporate-curated algorithm that is up to the task, especially when the primary purpose is to profit from advertising.

I think this is the root cause of the problem. Google can easily put a big dent in this problem by allowing users to create their own importable/exportable filters and support the dissemination of something like "EasyList for search results." But that kills their golden goose of advertising influence.

In before your comment ends up in QuickBooks search results for the rest of time :-)

> bring back the google from 5-10 years ago

Given Kagi's abysmal adoption rates, it's clear that good search isn't worth it for most people.

> You can't go back to the way things were.

> type in anything, and somehow it will read your mind

I think we can go back to the way things were, which had nothing to do with mind reading. In the past, you could type in word, and google would offer 10 million results, and you could page through each of them. That was very powerful, and google does not do that today.

I'm fine with this internet but the previous search.

Exactly. In a constantly changing world, you need constantly changing policy to achieve the same outcomes. Even then you probably won't replicate the past universe perfectly.

That subscription fee is just too big of an obstacle in a time when everything has a subscription and is still often degrading in quality. Seems like an unsolvable chicken and egg scenario though, since relying on advertising to make it free would just result in the same issues as everything else.

append ' before:2023' to your google searches.

I want the Internet from 5-10 years ago before Google incentivized this much SEO garbage. It wasn't awesome then but it was a lot better.

I was in high school 15 years ago and Google absolutely read minds to conclude Briney Spears was not a search for pickles but rather a pop artist. This was significant enough for them to come to go talk about it.

Google search degraded in usefulness before the panda update, when spammers had filled the web with low quality content designed to exploit Google's algorithms. Google improved their search to punish the content farms, and people were happy with that search for many years.

Google is remarkably effective at handling the scale. It doesn't seem up for handling the sheer army dedicated to misleading it. Especially now that they've been given tools for automating crap generation.

Ironically, Google itself was a key developer of that tech.

If there is any solution it would seem to involve removing the incentive to merely look at your page. That problem seems remarkably stubborn.

> "EasyList for search results"

Who will be in charge of curating that list? We know that crowd-sourced stuff is easily abused (see Amazon reviews, see YouTube comments).

I tried Kagi but just didn't see notably better results than other search engines. Maybe if I spent more time on the power user tools, or if Kagi offered more of a trial period I would have, but adding yet another monthly subscription is a high bar for me and what I saw didn't clear it.

These days my default assumption is that any SAAS product will get worse and more expensive over time, so it has to be pretty good to justify reworking my online habits around, given that I don't know how long I'll keep using it. Hopefully Kagi will be the exception to that rule, but I wouldn't bet on it.

It would indeed be crowd-sourced, but with a core set of maintainers. Wouldn't be all that different from EasyList or Steven Black's HOSTS file. They basically take in merge requests from the community and serve as an initial filter against garbage. [1]

And unlike Amazon reviews or YouTube comments, anyone can fork it if they think they can maintain it better.

[1] "The filter lists are currently maintained by four authors, Fanboy, MonztA, Khrin, Yuki2718 and PiQuark6046, who are ably assisted by an ample forum community." https://easylist.to/

I still search Google and other search engines from the command line. There is no "AI" garbage in the results. The way HN commenters refer to Google search in this thread, one might conclude it is not possible anymore to search the web without a popular browser running Javascript (which is a prerequisite for this "AI" stuff). That conclusion would be incorrect. It is still possible; I am still doing it every day.

It's quite literally this. It costs more than free and people don't want that. We're poor and poorer and everyone is overburdened by subscriptions for everything. I get that HN is in a rich bubble but most folks can't afford rent, food, and a search engine.

"seems like even some of the simplest searches are just packed with AI generated and SEO garbage"

I'll give you a concrete example of that and it is a right old pain.

Let's try upgrading Debian Bullseye to Bookworm. Search "upgrade debian bullseye to bookworm" - first hit from DDG is: https://www.debian.org/releases/bookworm/amd64/release-notes... - YES - Debian documentation, staid, verbose, stolid and correct.

Now let's try to upgrade a Raspberry Pi from Bullseye to Bookworm: Search "upgrade raspberry pi bullseye to bookworm". First hit: https://raspberrytips.com/upgrade-raspberry-pi-os-bookworm/.

There are loads more hits like the above and they are nearly all wrong. The RPI distribution is based on Debian Linux but has a few differences. Between those two versions of Debian, RPi changed things in /boot quite dramatically and failing to do that, you will end up with a weird chimera - I created several of these beasts until I fixed them: https://blog.scheib.me/2024/04/14/upgrade-raspberry-bullseye...

In this case it may actually be a blog matching the template of the AI clones! However, they do all look very similar.

What I really want is me from 5-10 years ago. When can we get a pill that will do that?

I was searching for a uniquely named company by exact name (think: verizon), and it was 80% of the way down the results page. Google knew exactly what I wanted to see and flooded my screen with alternatives who had paid them.

Agreed. That’s really my thoughts.

The internet or rather the crawlable WWW itself has changed.

The WWW content itself is no longer what it was.

You could probably take some pills that will make you think you are! LOL

I feel like testosterone therapy should be more readily available….

Do you think the proliferation of web frameworks makes it harder to crawl?

How do you do that?

You could get there somewhat. Do you exercise? Ready to put in some effort?

Probably has some influence. Because everything needs to be an app, implemented as an SPA, rather than just a web page. Hypertext is dead.

I don't think you know what you are asking. Do you really want 10 million pages of results, of which 99.999...% are SEO spam for Viagra et al, and on average you will need to browser ~9 million pages of results to find something that's actually "relevant"?

A close approximation might be a search-mode which penalizes results based on how many ads they have and how much of the page they cover...

This is how I do it

1. Make HTTP

2. Send HTTP via TCP

3. Perform text processing on the response body (I create own SERP instead of using Google's)

Personally, I use multiple programs, some I wrote myself in C, to perform these individual steps, connected by UNIX pipes and the shortest, simplest possible Bourne shell scripting

However there are countless ways to perform these steps in wide variety of programming languages; there is no need for UNIX or shell scripting, it is purely personal preference

>Think about the premise: type in anything, and somehow it will read your mind, intuit who you are and what you really wanted, find the exact thing amid the morass of the whole web, and then give it to you?

I never once asked for anything remotely like this. Maybe you could just show me results for the fucking thing I typed? When I go to the library, the Dewey decimal system doesn't rearrange itself based on all the metadata the library has on me and people fitting my demographic criteria, it just shows me what I fucking searched for.

Google does perfectly on the latter search. It returns a relevant blog post written by an actual human, and a bunch of forum threads about that exact upgrade path.

DDG has never been very good.

Were those organic results, or the paid ads that always appear first?

> costs more than free and people don't want that. We're poor and poorer and everyone is overburdened by subscriptions for everything

But that’s also the answer on preference. Google is good enough for most people. For everyone else, there can be a paid premium layer. Similar to news, this might be the equilibrium, not an anomaly.

i would love for pages to also be ranked by readability…

seems like local news sites are the absolute worst in this area

Search has degraded substantially in the past 5 years for reasons wholly, completely, and absolutely unrelated to SEO blogspam.

Google of 5 years ago didn't ignore words in almost every single query I made. Google of 5 years ago didn't constantly give me irrelevant garbage because they keep ignoring the words I use in my query.

This is a wholly separate issue from SEO crap. Ignoring search terms is 100% a Google issue and is 100% Google's fault!

Yes they did. It has been getting steadily worse since 2014 or so.