←back to thread

The man who killed Google Search?

(www.wheresyoured.at)
1884 points elorant | 1 comments | | HN request time: 0.211s | source
Show context
ot1138 ◴[] No.40135156[source]
Phenomenal article, very entertaining and aligns with my experience as a prominent search "outsider" (I founded the first search intelligence service back in 2004, which was later acquired by WPP. Do I have some stories).

The engineers at Google were wonderful to work with up to 2010. It was like a switch flipped mid-2011 and they became actively hostile to any third party efforts to monitor what they were doing. To put it another way, this would like NBC trying to sue Nielsen from gathering ratings data. Absurd.

Fortunately, the roadblocks thrown up against us were half-hearted ones and easily circumvented. Nevertheless, I had learned an important lesson about placing reliance for one's life work on a faceless mega tech corporation.

It was not soon after when Google eliminated "Don't Be Evil" from the mission statement. At least they were somewhat self aware, I suppose.

replies(6): >>40135263 #>>40135980 #>>40136233 #>>40138006 #>>40142360 #>>40146753 #
ChuckMcM ◴[] No.40135980[source]
I'm really glad the article came out though, it fills in some gaps that I was fairly confident about but didn't have anything other than my sense of the players and their actions to back up what I thought was going on.

I and a number of other people left in 2010. I went on to work at Blekko which was trying to 'fix' search using a mix of curation and ranking.

When I left, this problem of CPC's (the amount Google got per ad click in search) was going down (I believe mostly because of click fraud and advertisers losing faith in Google's metrics). While they were reporting it in their financial results, I had made a little spreadsheet[1] from their quarterly reports and you can see things tanking.

I've written here and elsewhere about it, and watched from the outside post 2010 and when people were saying "Google is going to steam roll everyone" I was saying, "I don't think so, I think unless they change they are dead already." There are lots of systemic reasons inside Google why it was hard for them to change and many of their processes reinforced the bad side of things rather than the good side. The question for me has always been "Will they pull their head out in time to recover?" recognizing that to do that they would have to be a lot more honest internally about their actions than they were when I was there. I was also way more pessimistic, figuring that they would be having company wide layoffs by 2015 to 2017 but they pushed that out by 5 years.

I remember pointing out to an engineering director in 2008 that Google was living in the dead husk of SGI[2] which caused them to laugh. They re-assured me that Google was here to stay. I pointed out that Wei Ting told me the same thing about SGI when they were building the campus. (SGI tried to recruit me from Sun which had a campus just down the road from where Google is currently.)

[1] https://docs.google.com/spreadsheets/d/18_y-Zyhx-5a1_kcW-x7p...

[2] Silicon Graphics -- https://www.sfchronicle.com/news/article/peninsula-high-tech...

replies(5): >>40136056 #>>40136121 #>>40136355 #>>40136434 #>>40140235 #
arromatic ◴[] No.40136056[source]
How did the slashtag feature worked and what did it do ? It seems like a interesting concept but sadly the site is dead . What happened to it ?
replies(1): >>40136422 #
ChuckMcM ◴[] No.40136422[source]
People would add sites for a particular topic (aka slashtag) to their list. That would build a virtual custom search engine within the search engine. And topic specific searches thrown at it would consistently out perform Bing and Google in terms of search quality. The meta "spam" slash tag (everyone got their own) would let you tell the engine sites you never wanted to see in your search results so if you were tired of your medical queries being spammed by quacks, add them to your spam list and they wouldn't be in your results.
replies(2): >>40138297 #>>40141778 #
arromatic ◴[] No.40141778[source]
Why did it shut down ?
replies(1): >>40182525 #
ChuckMcM ◴[] No.40182525[source]
Ultimately, lack of traffic. During Blekko's lifetime Google went from paying people less than $10M/quarter to send their search traffic to Google to over $4B/quarter to do that. If you are ad based you need traffic to serve ads to.

At some point a pay for search model might emerge that has a big enough audience to support a company but that time is not yet here.

replies(1): >>40237696 #
arromatic ◴[] No.40237696[source]
1 . Does that mean blekko was something similar to millionshort ? 2. Was blekko capable of tackling seo sites or blogspam taht we have today or it had the advantage of low spam site count from the old web ? 3. Does it have a chance of coming back like how yahoo has been recently hinting a comeback ? 4. A stupid question : How much will it cost it build a blekko today ?
replies(1): >>40238105 #
1. ChuckMcM ◴[] No.40238105[source]
Not sure on #1, definitely mitigated SEO sites and blogspam (on an individual basis, if you added a spammy blog to your personal spam list it stopped showing up in results). As a result on slashtag searches there was very little spam.

Would it come back? As it was? no. The folks at Bing used some of the techniques to mitigate some spam in Bing results but didn’t implement slashtags.

It would cost between $3 - $6M to go from scratch to developing a 3 billion page index with a 10 billion page crawl ‘frontier’. You can seed the crawl with Common Crawl. If you can get $10 RPM’s ($10 per thousand queries) and roughly 10M queries/day (so $10k/day recurring revenue) you can run an operationally cash flow positive business. You would want to grow it organically to a 10 billion page index on a 100 billion page crawl which would cover 90+% of the english language queries. With clever optimizations (like a news sieve to only index pages about the news that made ‘sense’) you might improve efficiencies. You would also want to focus on reference applications (people who use search to get their job done) for paid subscriber growth, and simpler commercial partnerships for managing ad lead generation on commercial search (people looking for products or services).

Also you would need to be an advertising ‘primary’ (not taking feeds from networks on a revenue share model) So, for example, working directly with Amazon to both efficiently access their internal product index and to surface it on commercial queries. Note people like Amazon do their own advertising business on their own index so you compete with that to some extent and navigating that early is essential.

Certainly doable but not something that your typical venture fund would go for. It would have a longer payback time (lower internal rate of return) than VC’s look for.