←back to thread

Microsoft and OpenAI's close partnership shows signs of fraying

(www.nytimes.com)
321 points jhunter1016 | 2 comments | 18 Oct 24 11:11 UTC | HN request time: 0s | source
Show context
Roark66 ◴[18 Oct 24 12:02 UTC] No.41878594[source]▶
>OpenAI plans to loose $5 billion this year

Let that sink in for anyone that has incorporated Chatgpt in their work routines to the point their normal skills start to atrophy. Imagine in 2 years time OpenAI goes bust and MS gets all the IP. Now you can't really do your work without ChatGPT, but it cost has been brought up to how much it really costs to run. Maybe $2k per month per person? And you get about 1h of use per day for the money too...

I've been saying for ages, being a luditite and abstaining from using AI is not the answer (no one is tiling the fields with oxen anymore either). But it is crucial to at the very least retain 50% of capability hosted models like Chatgpt offer locally.

replies(20): >>41878631 #>>41878635 #>>41878683 #>>41878699 #>>41878717 #>>41878719 #>>41878725 #>>41878727 #>>41878813 #>>41878824 #>>41878984 #>>41880860 #>>41880934 #>>41881556 #>>41881938 #>>41882059 #>>41883046 #>>41883088 #>>41883171 #>>41885425 #
sebzim4500 ◴[18 Oct 24 12:20 UTC] No.41878719[source]▶
The marginal cost of inference per token is lower than what OpenAI charges you (IIRC about 2x cheaper), they make a loss because of the enormous costs of R&D and training new models.
replies(4): >>41878823 #>>41878875 #>>41878927 #>>41879029 #
ignoramous ◴[18 Oct 24 12:41 UTC] No.41878875[source]▶
> The marginal cost of inference per token is lower than what OpenAI charges you

Unlike most Gen AI shops, OpenAI also incurs a heavy cost for traning base models gunning for SoTA, which involves drawing power from a literal nuclear reactor inside data centers.

replies(2): >>41878936 #>>41878996 #
candiddevmike ◴[18 Oct 24 12:55 UTC] No.41878996{3}[source]▶
> literal nuclear reactor inside data centers

This is fascinating to think about. Wonder what kind of shielding/environmental controls/all other kinds of changes you'd need for this to actually work. Would rack-sized SMR be contained enough not to impact anything? Would datacenter operators/workers need to follow NRC guidance?

replies(3): >>41880937 #>>41881030 #>>41882203 #
1. talldayo ◴[18 Oct 24 16:27 UTC] No.41880937{4}[source]▶
I think the simple answer is that it doesn't make sense. Nuclear power plants generate a byproduct that inherently limits the performance of computers; heat. Having either a cooling system, reactor or turbine located inside a datacenter is immediately rendered pointless because you end up managing two competing thermal systems at once. There is no reason to localize a reactor inside a datacenter when you could locate it elsewhere and pipe the generated electricity into it via preexisting high voltage lines.
replies(1): >>41882190 #
2. kergonath ◴[18 Oct 24 18:35 UTC] No.41882190[source]▶
> Nuclear power plants generate a byproduct that inherently limits the performance of computers; heat.

The reactor does not need to be in the datacenter. It can be a couple hundreds meters away, bog-standard cables would be perfectly able to move the electrons. The cables being 20m or 200m long does not matter much.

You’re right though, putting them in the same building as a datacenter still makes no sense.