Most active commenters
  • noosphr(4)
  • fragmede(3)

←back to thread

Google is winning on every AI front

(www.thealgorithmicbridge.com)
993 points vinhnx | 34 comments | 12 Apr 25 03:58 UTC | HN request time: 1.641s | source | bottom
Show context
thunderbird120 ◴[12 Apr 25 06:13 UTC] No.43661807[source]▶
This article doesn't mention TPUs anywhere. I don't think it's obvious for people outside of google's ecosystem just how extraordinarily good the JAX + TPU ecosystem is. Google several structural advantages over other major players, but the largest one is that they roll their own compute solution which is actually very mature and competitive. TPUs are extremely good at both training and inference[1] especially at scale. Google's ability to tailor their mature hardware to exactly what they need gives them a massive leg up on competition. AI companies fundamentally have to answer the question "what can you do that no one else can?". Google's hardware advantage provides an actual answer to that question which can't be erased the next time someone drops a new model onto huggingface.

[1]https://blog.google/products/google-cloud/ironwood-tpu-age-o...

replies(12): >>43661870 #>>43661974 #>>43663154 #>>43663455 #>>43663647 #>>43663720 #>>43663956 #>>43664320 #>>43664354 #>>43672472 #>>43673285 #>>43674134 #
1. noosphr ◴[12 Apr 25 06:29 UTC] No.43661870[source]▶
And yet google's main structural disadvantage is being google.

Modern BERT with the extended context has solved natural language web search. I mean it as no exaggeration that _everything_ google does for search is now obsolete. The only reason why google search isn't dead yet is that it takes a while to index all web paged into a vector database.

And yet it wasn't google that released the architecture update, it was hugging face as a summer collaboration between a dozen people. Google's version came out in 2018 and languished for a decade because it would destroy their business model.

Google is too risk averse to do anything, but completely doomed if they don't cannibalize their cash cow product. Web search is no longer a crown jewel, but plumbing that answering services, like perplexity, need. I don't see google being able to pull off an iPhone moment where they killed the iPod to win the next 20 years.

replies(7): >>43661911 #>>43661929 #>>43662090 #>>43662277 #>>43662527 #>>43662862 #>>43670870 #
2. visarga ◴[12 Apr 25 06:39 UTC] No.43661911[source]▶
> Modern BERT with the extended context has solved natural language web search. I mean it as no exaggeration that _everything_ google does for search is now obsolete.

The web UI for people using search may be obsolete, but search is hot, all AIs need it, both web and local. It's because models don't have recent information in them and are unable to reliably quote from memory.

replies(2): >>43661998 #>>43662189 #
3. podnami ◴[12 Apr 25 06:43 UTC] No.43661929[source]▶
Do we have insights on whether they knew that their business model was at risk? My understanding is that OpenAI’s credibility lies in seeing the potential of scaling up a transformer-based model and that Google was caught off guard.
4. nroets ◴[12 Apr 25 06:55 UTC] No.43661998[source]▶
And models often makes reasoning errors. Many users will want to check that the sources substantiate the conclusion.
replies(1): >>43679498 #
5. dash2 ◴[12 Apr 25 07:14 UTC] No.43662090[source]▶
They can just plug the google.com web page into their AI. They already do that.
replies(1): >>43663664 #
6. vidarh ◴[12 Apr 25 07:29 UTC] No.43662189[source]▶
The point is that the secret sauce in Google's search was better retrieval, and the assertion above is that the advantage there is gone. While crawling the web isn't a piece of cake, it's a much smaller moat than retrieval quality was.
replies(1): >>43668005 #
7. petesergeant ◴[12 Apr 25 07:46 UTC] No.43662277[source]▶
> Google is too risk averse to do anything, but completely doomed if they don't cannibalize their cash cow product.

Google's cash-cow product is relevant ads. You can display relevant ads in LLM output or natural language web-search. As long as people are interacting with a Google property, I really don't think it matters what that product is, as long as there are ad views. Also:

> Web search is no longer a crown jewel, but plumbing that answering services, like perplexity, need

This sounds like a gigantic competitive advantage if you're selling AI-based products. You don't have to give everyone access to the good search via API, just your inhouse AI generator.

replies(2): >>43662697 #>>43663002 #
8. danpalmer ◴[12 Apr 25 08:31 UTC] No.43662527[source]▶
This would be like claiming in 2010 that because Page Rank is out there, search is a solved problem and there’s no secret sauce, and the following decade proved that false.
replies(1): >>43670304 #
9. michaelt ◴[12 Apr 25 09:07 UTC] No.43662697[source]▶
Kodak was well placed to profit from the rise of digital imaging - in the late 1970s and early 1980s Kodak labs pioneered colour image sensors, and was producing some of the highest resolution CCDs out there.

Bryce Bayer worked for Kodak when he invented and patented the Bayer pattern filter used in essentially every colour image sensor to this day.

But the problem was: Kodak had a big film business - with a lot of film factories, a lot of employees, a lot of executives, and a lot of recurring revenue. And jumping into digital with both feet would have threatened all that.

So they didn't capitalise on their early lead - and now they're bankrupt, reduced to licensing their brand to third-party battery makers.

> You can display relevant ads in LLM output or natural language web-search.

Maybe. But the LLM costs a lot more per response.

Making half a cent is very profitable if you only take 0.2s of CPU to do it. Making half a cent with 30 seconds multiple GPUs, consuming 1000W of power... isn't.

replies(3): >>43663362 #>>43663969 #>>43665437 #
10. jampekka ◴[12 Apr 25 09:34 UTC] No.43662862[source]▶
> Modern BERT with the extended context has solved natural language web search.

I doubt this. Embedding models are no panacea even with a lot simpler retrieval tasks like RAG.

replies(1): >>43670314 #
11. lonelyasacloud ◴[12 Apr 25 10:01 UTC] No.43663002[source]▶
> Google's cash-cow product is relevant ads.

As a business Google's interest is in showing ads that make it the most money - if they quickly show just the relevant information then Google loses advertising opportunities.

To an extent, it is the web equivalent of irl super markets intentionally moving stuff around and having checkout displays.

replies(1): >>43663083 #
12. dambusm ◴[12 Apr 25 10:20 UTC] No.43663083{3}[source]▶
> As a business Google's interest is in showing ads that make it the most money - if they quickly show just the relevant information then Google loses advertising opportunities.

This is just a question of UX- the purpose of their search engine was already to show the most relevant information (ie. links), but they just put some semi-relevant information (ie. sponsored links) first, and make a fortune. They can just do the same with AI results.

13. djtango ◴[12 Apr 25 11:06 UTC] No.43663362{3}[source]▶
This is a good anecdote and it reminds me of how Sony had cloud architecture/digital distribution, a music label, movie studio, mobile phones, music players, speakers, tvs, laptops, mobile apps... and totally missed out on building Spotify or Netflix.

I do think Google is a little different to Kodak however; their scale and influence is on another level. GSuite, Cloud, YouTube and Android are pretty huge diversifications from Search in my mind even if Search is still the money maker...

replies(2): >>43663684 #>>43663737 #
14. fragmede ◴[12 Apr 25 12:02 UTC] No.43663664[source]▶
but because users are used to doing that for free, they can't charge money for that, but if they don't charge money for that, and no one's seeing ads, then where does they money come from?
replies(1): >>43665059 #
15. fragmede ◴[12 Apr 25 12:05 UTC] No.43663684{4}[source]▶
It goes to internal corporate culture, and what happens to you when you point out an uncomfortable truth. Do we shoot the messenger, or heed her warnings and pivot the hopefully not Titanic? RIM/Blackberry didn't manage to avoid it either.

People like to believe CEOs aren't worth their pay package, and sometimes they're not. But a look at a couple of their failures and a different CEO of Kodak wouldn't have had what happened happen, makes me think that sometimes, some of them do deserve that.

replies(2): >>43664137 #>>43679541 #
16. decimalenough ◴[12 Apr 25 12:16 UTC] No.43663737{4}[source]▶
Sony's Achilles heel was and remains software. You can't build a Spotify or Netflix if you can't build a proper website.
replies(1): >>43664285 #
17. dgacmu ◴[12 Apr 25 12:56 UTC] No.43663969{3}[source]▶
1/2 kW/minute costs about $0.001 so you technically could make a profit at that rate. The real problem is the GPU cost - a $20k GPU amortized over five years costs $0.046 per second. :)
replies(1): >>43666278 #
18. johnecheck ◴[12 Apr 25 13:20 UTC] No.43664137{5}[source]▶
If the king/ceo is great, autocracy works well.

When a fool inevitably takes the throne, disaster ensues.

I can't say for sure that a different system of government would have saved Kodak. But when one man's choices result in disaster for a massive organization, I don't blame the man. I blame the structure that laid the power to make such a mistake on his shoulders.

replies(1): >>43667384 #
19. vel0city ◴[12 Apr 25 13:40 UTC] No.43664285{5}[source]▶
That, and while Sony had all these big groups they often didn't play nice with each other. Look at how they failed to make Minidisc into any useful data platforms with PCs, largely because MD's were consumer devices and not personal computers so they were pretty much only seen as music hardware.

Even on the few Vaios that had MD drives on them, they're pretty much just an external MD player permanently glued to the device instead of being a full and deeply integrated PC component.

20. eitally ◴[12 Apr 25 15:10 UTC] No.43665059{3}[source]▶
Well, it clearly affects search ads, but in terms of revenue streams Google is already somewhat diversified:

1. Search ads (at risk of disintermediation) 2. Display ads (not going anywhere) 3. Ad-supported YouTube 4. Ad-supported YouTube TV 5. Ad-supported Maps 6. Partnership/Ad supported Travel, YouTube, News, Shopping (and probably several more) 7. Hardware (ChromeOS licensing, Android, Pixel, Nest) 8. Cloud

There are probably more ad-supported or ad-enhanced properties, but what's been shifting over the past few years is the focus on subscription-supported products:

1. YouTube TV 2. YouTube Premium 3. GoogleOne (initially for storage, but now also for advanced AI access) 4. Nest Aware 5. Android Play Store 6. Google Fi 7. Workspace (and affiliated products)

In terms of search, we're already seeing a renaissance of new options, most of which are AI-powered or enhanced, like basic LLM interfaces (ChatGPT, Gemini, etc), or fundamentally improved products like Perplexity & Kagi. But Google has a broad and deep moat relative to any direct competitors. Its existential risk factors are mostly regulation/legal challenge and specific product competition, but not everything on all fronts all at once.

21. ◴[12 Apr 25 15:52 UTC] No.43665437{3}[source]▶
22. pingou ◴[12 Apr 25 17:18 UTC] No.43666278{4}[source]▶
How do you get that? I get $0.0001 per second over 5 years to reach 20k.
replies(1): >>43666426 #
23. dgacmu ◴[12 Apr 25 17:38 UTC] No.43666426{5}[source]▶
Because I'm an idiot and left off a factor of 365. Thank you! A 20k GPU for 30 seconds is 1/3 of a cent. Still more than the power but also potentially profitable under this scenario informing all the other overhead and utilization.
24. fragmede ◴[12 Apr 25 19:43 UTC] No.43667384{6}[source]▶
that seems weird. Why hold up one person as being great while not also holding up one person as not? If my leader led me into battle and we were victorious, we'd put it on them. if they lead us to ruin, why should I blame the organizational structure that led to them getting power as the culprit instead of blaming them directly?
replies(1): >>43680915 #
25. pixl97 ◴[12 Apr 25 21:17 UTC] No.43668005{3}[source]▶
Eh, I don't really see that.

Crawling the web has a huge moat because a huge number of sites have blocked 'abusive' crawlers except Google and possibly Bing.

For example just try to crawl sites like Reddit and see how long before you're blocked and get a "please pay us for our data" message.

replies(2): >>43671333 #>>43675979 #
26. noosphr ◴[13 Apr 25 05:27 UTC] No.43670304[source]▶
In a time where statistical models couldn't understand natural language the click stream from users was their secret sauce.

Today a consumer grade >8b decoder only model does a better job of predicting if some (long) string of text matches a user query than any bespoke algorithm would.

The only reason why encoder only models are better than decoder only models is that you can cache the results against the corpus ahead of time.

27. noosphr ◴[13 Apr 25 05:30 UTC] No.43670314[source]▶
RAG is literally what Google Search is.

Unlike the natural language queries that RAG has to deal with, Google searches are (usually) atomic ideas and encoder-only models have a much easier time with them.

28. marsten ◴[13 Apr 25 07:41 UTC] No.43670870[source]▶
I think what may save Google from an Innovator's Dilemma extinction is that none of the AI would-be Google killers (OpenAI etc.) have figured out how to achieve any degree of lock-in. We're in a phase right now where everybody gets excited by the latest model and the switching cost is next to zero. This is very different from the dynamics of, say, Intel missing the boat on mobile CPUs.

I've been wondering for some time what sustainable advantage will end up looking like in AI. The only obvious thing is that whoever invents an AI that can remember who you are and every conversation it's had with you -- that will be a sticky product.

replies(1): >>43678157 #
29. literalAardvark ◴[13 Apr 25 09:16 UTC] No.43671333{4}[source]▶
My experience running a few hundred very successful shops (hundreds of thousands of orders per month) is that there's no need for quotes around 'abusive'.

95% of our load is from crawlers, so we have to pick who to serve.

If they want our data all they need to do is offer a way for us to send it, we're happy to increase exposure and shopping aggregation site updates are our second highest priority task after price and availability updates.

30. vidarh ◴[13 Apr 25 21:26 UTC] No.43675979{4}[source]▶
It may be tricky, but it's a piece of cake compared to doing good retrieval.
31. noosphr ◴[14 Apr 25 04:51 UTC] No.43678157[source]▶
Who ever gets AI to be able to search the whole corpus of human knowledge. I'm not just talking about web pages, I'm talking every book, every scientific paper, every news paper, every piece of text stored somewhere.

I've build RAG systems that index tokens in the 1e12 range and the main thing stopping us from having a super search that will make google look like the library card catalogue is the copyright system.

A country that ignores that and builds the first XXX billion parameter encoder only model will do for knowledge work what the high pressure steam engine did for muscle work.

32. fennecfoxy ◴[14 Apr 25 09:16 UTC] No.43679498{3}[source]▶
As they should do even for a Google search.

I see search engines as a dripfeed from a firehose, not some magical thing that's going to get me the 100% correct 100% accurate result.

Humans are the most prolific liars; I could never trust search results anyway since Google may find something that looks right but the author may be heavily biased, uninformed and all manner of other things anyways.

33. fennecfoxy ◴[14 Apr 25 09:23 UTC] No.43679541{5}[source]▶
I firmly believe the majority of CEOs and executives may do something useful (often not) but none of them truly _earn_ their multi-million salaries (for those that are on the modern 100-1000x salaries). It's just suits, handshakes and social connections from certain schools/families. That's all.

Constantly I see them dodging responsibility or resigning (as an "apology") during a crisis they caused and then moving on to the next place they got buddies at for another multi-mil salary.

Many here would defend 'em tho. HN/SV tech people seem to aspire to such things from what I've seen. The rest of us just really think computers are super cool.

34. johnecheck ◴[14 Apr 25 13:13 UTC] No.43680915{7}[source]▶
I'm not saying you'd be wrong to blame the bad leader - just that blaming them doesn't achieve much.

The CEO takes the blame, the board picks a new one (Unless the CEO has special shares that make them impossible to dismiss), and we go on hoping that the king isn't an idiot this time.

My reading of history is that some people are fools - we can blame them for their incompetence or we can set out to build foolproof systems. (Obviously, nothing will be truly foolproof. But we can build systems that are robust against a minority of the population being fools/defectors.)