Most active commenters
  • runako(6)

←back to thread

ChatGPT's enterprise success against Copilot fuels OpenAI/Microsoft rivalry

(www.bloomberg.com)
302 points mastermaq | 24 comments | 24 Jun 25 16:02 UTC | HN request time: 0.427s | source | bottom
Show context
hbn ◴[24 Jun 25 20:32 UTC] No.44370715[source]▶
My work gives us Copilot access since We're a Microsoft Shop™

It's too bad Copilot is by far the dumbest competitor in the space

My favorite interaction so far was when I prompted it with:

    ffmpeg command to convert movie.mov into a reasonably sized mp4
Sure, it's not the most direction instructions, but I tend to give it just enough to get the job done, assuming the LLM knows what its purpose is as an LLM, and it always works with the other chatbots.

Copilot's response:

    I implemented and executed the Python code above to convert movie.mov to a reasonably sized movie.mp4 using ffmpeg.
    
    However, the Python code failed since it was not able to find and access movie.mov file.
    Do you want me to try again or is there anything else that I can help you with?
Note that I didn't cut anything out. It didn't actually provide me any "Python code above"
replies(22): >>44370829 #>>44371002 #>>44371022 #>>44371053 #>>44371065 #>>44371287 #>>44371335 #>>44371358 #>>44371628 #>>44371891 #>>44371914 #>>44371978 #>>44372301 #>>44372892 #>>44373260 #>>44373493 #>>44373864 #>>44374419 #>>44374747 #>>44376761 #>>44377612 #>>44379849 #
NordSteve ◴[24 Jun 25 21:38 UTC] No.44371358[source]▶
I pasted your prompt:

  ffmpeg command to convert movie.mov into a reasonably sized mp4
into the Copilot app just now. Here's the response:

  ffmpeg -i movie.mov -vcodec libx264 -crf 23 -preset medium -acodec aac -b:a 128k movie_converted.mp4
with this explanation

  Here’s the breakdown:
  - -vcodec libx264: Uses the H.264 video codec, great for compression and compatibility.
  - -crf 23: Constant Rate Factor — lower means better quality and larger size; higher means smaller size but lower quality. 23 is a nice middle ground.
  - -preset medium: Controls encoding speed vs. compression efficiency. You can try slow or faster depending on your needs.
  - -acodec aac -b:a 128k: Compresses the audio using AAC at 128 kbps — pretty standard and compact.
  If you're looking for even smaller file sizes, try bumping -crf up to 26 or using -preset slow.
  Want to bake in scaling, remove metadata, or trim the video too? Happy to tailor the command further.
replies(5): >>44371528 #>>44372925 #>>44373007 #>>44374040 #>>44376405 #
Aurornis ◴[25 Jun 25 01:56 UTC] No.44373007[source]▶
I did the same thing for several iterations and all of the responses were equally helpful.

We get these same anecdotes about terrible AI answers frequently in a local Slack I’m in. I think people love to collect them as proof that AI is terrible and useless. Meanwhile other people have no problem hitting the retry button and getting a new answer.

Some of the common causes of bad or weird responses that I’ve learned from having this exact same conversation over and over again:

- Some people use one never-ending singular session with Copilot chat, unaware that past context is influencing the answer to their next question. This is a common way to get something like Python code in response to a command line question if you’re in a Python project or you’ve been asking Python questions.

- They have Copilot set to use a very low quality model because they accidentally changed it, or they picked a model they thought was good but is actually a low-cost model meant for light work.

- They don’t realize that Copilot supports different models and you have to go out of your way to enable the best ones.

AI discussions are weird because there are two completely different worlds of people using the same tools. Some people are so convinced the tool will be bad that they give up at the slightest inconvenience or they even revel in the bad responses as proof that AI is bad. The other world spends some time learning how to use the tools and work with a solution that doesn’t always output the right answer.

We all know AI tools are not as good as the out of control LinkedIn influencer hype, but I’m also tired of the endless claims that the tools are completely useless.

replies(9): >>44373253 #>>44373471 #>>44373558 #>>44373634 #>>44373822 #>>44375598 #>>44375719 #>>44376790 #>>44377124 #
1. muppetman ◴[25 Jun 25 04:34 UTC] No.44373634[source]▶
The "pick your model" thing is so stupid.

"How dumb do you want your AI to be?"

"Why do I have to select?"

"Because smart costs money"

"So... I can have dumb AI but it's cheaper?"

"Yes"

"How would the average person know which to pick?"

"Oh you can't know."

I hope they can invent an AI that knows which AI model my question should target cheaply.

replies(7): >>44373707 #>>44374320 #>>44374834 #>>44376182 #>>44376980 #>>44377115 #>>44380649 #
2. reilly3000 ◴[25 Jun 25 04:50 UTC] No.44373707[source]▶
I think you make a good point. Cursor is doing a basic “auto” model selection feature and it could probably get smarter, but to gauge the complexity of the response you might need to run it first. You could brute force it with telemetry and caching if you can trust the way you measure success.
3. HappMacDonald ◴[25 Jun 25 06:54 UTC] No.44374320[source]▶
You bring up the important point that for a company who earns money off of tokens wasted, a confusing selection of models can translate into extra spend to experiment with tweaking them.

Some users may not appreciate that, but many more might be drawn to the "adjust the color balance on the TV" vibes.

4. addandsubtract ◴[25 Jun 25 08:18 UTC] No.44374834[source]▶
> I hope they can invent an AI that knows which AI model my question should target cheaply.

Isn't that the idea of OpenRouter?

replies(1): >>44375337 #
5. oersted ◴[25 Jun 25 09:38 UTC] No.44375337[source]▶
Not exactly, but yeah. OpenRouter is a unified API, directory and billing system for LLM providers.

I think you are getting confused by the term "Model Routing", which to be fair OpenRouter does support, but it's a secondary feature and it's not their business focus. Actually OpenRouter is more focused on helping you choose the best provider for a specific open model based on their history of price, speed, reliability, privacy...

The model routing is simply provided by NotDiamond.ai, there are a number of other startups in this space.

https://openrouter.ai/docs/features/model-routing

6. prepend ◴[25 Jun 25 11:48 UTC] No.44376182[source]▶
I imagine that we need a bootstrap ai to help you optimize the right ai for each task.

I don’t think I’d trust the vendor’s ai to optimize when they will likely bias toward revenue. So a good case for a local ai that only has my best interests at heart.

Currently, the guidance from vendors is “try it and see which yields the best results” which is kind of like “buy this book, read it, and see if you like it” and how of course the publisher wants you to take this action because they get their money.

7. setopt ◴[25 Jun 25 13:16 UTC] No.44376980[source]▶
> I hope they can invent an AI that knows which AI model my question should target cheaply.

It would be great to have a cheap AI that can self-evaluate how confident it is in its reply, and ask its expensive big brother for help automatically when it’s not.

replies(1): >>44377061 #
8. taikahessu ◴[25 Jun 25 13:24 UTC] No.44377061[source]▶
That would actually be the AGI we are waiting for, since we - as humans, in surprisingly big portion of all cases - don't know how or can't seem to do that either!
replies(1): >>44385297 #
9. runako ◴[25 Jun 25 13:30 UTC] No.44377115[source]▶
And then the model names & descriptions are virtually useless at providing any guidance.

ChatGPT lets me choose between GPT-4o ("Great for most tasks"), o3 ("Uses advanced reasoning"), o4-mini ("Fastest at advanced reasoning"), and o4-mini-high ("Great at coding and visual reasoning").

Is what I'm doing "most tasks"? How do I know when I want "advanced reasoning"? Great, I want advanced reasoning, so I should choose the faster one with the higher version number, right? etc.

replies(3): >>44377211 #>>44378301 #>>44378654 #
10. dataflow ◴[25 Jun 25 13:38 UTC] No.44377211[source]▶
Can't you just run a few examples by hand to see how they perform for your tasks, before committing to any for production?
replies(1): >>44377896 #
11. runako ◴[25 Jun 25 14:40 UTC] No.44377896{3}[source]▶
> before committing to any for production

I'm talking about ChatGPT, which is a Web and desktop app where users run interactive sessions. What does "production" mean in this sense?

replies(2): >>44378495 #>>44379891 #
12. antonkochubey ◴[25 Jun 25 15:15 UTC] No.44378301[source]▶
Then there's GPT-4.5 which is "Good for writing and exploring ideas" (are the other models bad for this?), and GPT-4.1 which is "Great for quick coding and analysis" (is a model which "uses advanced reasoning" not great for these things?)
13. dataflow ◴[25 Jun 25 15:34 UTC] No.44378495{4}[source]▶
I think I misunderstood what people were talking about. Somehow I thought it was about their APIs, for specific uses in other apps.
replies(1): >>44379178 #
14. tzs ◴[25 Jun 25 15:49 UTC] No.44378654[source]▶
Can you describe your task and then ask ChatGPT which model you should use?
replies(1): >>44379154 #
15. runako ◴[25 Jun 25 16:33 UTC] No.44379154{3}[source]▶
This presents the same problem, since none of the models are indicated to be best at choosing the model to use for a task.
replies(1): >>44379880 #
16. runako ◴[25 Jun 25 16:34 UTC] No.44379178{5}[source]▶
To their credit, they did get this part correct. "ChatGPT" is the user-facing apps. The models have terrible names that do not include "ChatGPT".

Anthropic, by contrast, uses the same name for the user-facing app and the models. This is confusing, because the user-facing apps have capabilities not native to the models themselves.

17. hluska ◴[25 Jun 25 17:38 UTC] No.44379880{4}[source]▶
Try different ones out and learn which works best for what type of work?
replies(1): >>44380730 #
18. hluska ◴[25 Jun 25 17:39 UTC] No.44379891{4}[source]▶
It’s simple - practice using them instead of complaining. Maybe you’ll figure out the differences on your own.
replies(2): >>44379979 #>>44380708 #
19. BobaFloutist ◴[25 Jun 25 17:47 UTC] No.44379979{5}[source]▶
Imagine if this is what people suggested when I asked what kind of screwdriver I should use for a given screw, because they're all labelled, like, "Phillips. Phillips 2.0. Phillips.2.second. Phillips.2.second.version 2.0. Phillips Head Screwdriver. Phillips.2.The.Second.Version. Phillips.2.the.second.Version 2.0"
20. bastard_op ◴[25 Jun 25 18:49 UTC] No.44380649[source]▶
I usually feel with chatgpt picking a model is like "Which of the three stooges would you like to talk to, curly, larry, or moe (or worse, curly joe)?" I usually only end up using o3 because gpt-40 is just that bad, so why would I ever want to talk to a lesser stooge?

If paying by API use it probably makes more sense to talk to a lesser stooge where possible, but for a standard pro plan I just find the lesser models aren't worth the time to use in frustration they cause.

21. runako ◴[25 Jun 25 18:54 UTC] No.44380708{5}[source]▶
As a person who uses LLMs daily, I do in fact do this. Couple problems with this approach:

- there are billions of people who are not accustomed to using software this way, who are in the expected target market for this software. Most people cannot tell you the major version number of their mobile OS.

- this approach requires each individual to routinely perform experiments with the expanding firmament of models and versions. This is obviously user-hostile.

Anyway, my hot take here is that making things easier for users is better. I understand that is controversial on this site.

22. runako ◴[25 Jun 25 18:57 UTC] No.44380730{5}[source]▶
Without getting too much into semantics, I would suspect that most individuals would have trouble classifying their "type of work" against an opaque set of "type of work" classifiers buried in a model.
replies(1): >>44380977 #
23. elbear ◴[25 Jun 25 19:23 UTC] No.44380977{6}[source]▶
He was suggesting that you try different models for the same thing and see which output you like best. It's tedious but at least you get an answer.
24. setopt ◴[26 Jun 25 08:16 UTC] No.44385297{3}[source]▶
On the other hand, ChatGPT seems to be getting better at knowing when it should Google something for me rather than hallucinate something.

Shouldn’t asking a more expensive model for input be a similar level of «tool use»?