OpenAI o3 and o4-mini

(openai.com)

Show context

georgewsinger ◴[16 Apr 25 17:20 UTC] No.43707951[source]▶

Very impressive! But under arguably the most important benchmark -- SWE-bench verified for real-world coding tasks -- Claude 3.7 still remains the champion.[1]

Incredible how resilient Claude models have been for best-in-coding class.

[1] But by only about 1%, and inclusive of Claude's "custom scaffold" augmentation (which in practice I assume almost no one uses?). The new OpenAI models might still be effectively best in class now (or likely beating Claude with similar augmentation?).

replies(7): >>43708008 #>>43708068 #>>43708249 #>>43708545 #>>43709203 #>>43713202 #>>43716307 #

1. pizzathyme ◴[16 Apr 25 19:10 UTC] No.43709203[source]▶

>>43707951 #

The image generation improvement with o4-mini is incredible. Testing it out today, this is a step change in editing specificity even from the ChatGPT 4o LLM image integration just a few weeks ago (which was already a step change). I'm able to ask for surgical edits, and they are done correctly.

There isn't a numerical benchmark for this that people seem to be tracking but this opens up production-ready image use cases. This was worth a new release.

replies(3): >>43710367 #>>43710556 #>>43711280 #

2. mchusma ◴[16 Apr 25 21:04 UTC] No.43710367[source]▶

>>43709203 (TP) #

Thanks for sharing that. that was more interesting then their demo. I tried it and it was pretty good! I have felt that the ability to iterate from images blocked this from any real production use I had. This may be good enough now.

Example of edits (not quite surgical but good): https://chatgpt.com/share/68001b02-9b4c-8012-a339-73525b8246...

replies(1): >>43712143 #

3. ilaksh ◴[16 Apr 25 21:23 UTC] No.43710556[source]▶

>>43709203 (TP) #

wait, o4-mini outputs images? What I thought I saw was the ability to do a tool call to zoom in on an image.

Are you sure that's not 4o?

replies(1): >>43711306 #

4. Agentus ◴[16 Apr 25 23:08 UTC] No.43711280[source]▶

>>43709203 (TP) #

also another addition: i previously tried to upload an image for chatgpt to edit and it was incapable under the previous model i tried. Now its able to change uploaded images using o4mini.

5. AaronAPU ◴[16 Apr 25 23:11 UTC] No.43711306[source]▶

>>43710556 #

I’m generating logo designs for merch via o4-mini-high and they are pretty good. Good text and comprehending my instructions.

replies(2): >>43713173 #>>43713182 #

6. ec109685 ◴[17 Apr 25 01:24 UTC] No.43712143[source]▶

>>43710367 #

I don’t know if they let you share the actual images when sharing a chat. For me, they are blank.

7. ilaksh ◴[17 Apr 25 04:38 UTC] No.43713173{3}[source]▶

>>43711306 #

in the api or on the website?

8. ilaksh ◴[17 Apr 25 04:39 UTC] No.43713182{3}[source]▶

>>43711306 #

It's using the new gpt-4o, a version that's not in the API

↑