←back to thread

Gemini CLI

(blog.google)
1348 points sync | 3 comments | | HN request time: 0.606s | source
Show context
joelm ◴[] No.44379446[source]
Been using Claude Code (4 Opus) fairly successfully in a large Rust codebase, but sometimes frustrated by it with complex tasks. Tried Gemini CLI today (easy to get working, which was nice) and it was pretty much a failure. It did a notably worse job than Claude at having the Rust code modifications compile successfully.

However, Gemini at one point output what will probably be the highlight of my day:

"I have made a complete mess of the code. I will now revert all changes I have made to the codebase and start over."

What great self-awareness and willingness to scrap the work! :)

replies(8): >>44379714 #>>44380383 #>>44380768 #>>44380866 #>>44381146 #>>44381754 #>>44383245 #>>44386866 #
ZeroCool2u ◴[] No.44379714[source]
Personally my theory is that Gemini benefits from being able to train on Googles massive internal code base and because Rust has been very low on uptake internally at Google, especially since they have some really nice C++ tooling, Gemini is comparatively bad at Rust.
replies(5): >>44380405 #>>44380865 #>>44381697 #>>44382948 #>>44383662 #
thimabi ◴[] No.44381697[source]
> Personally my theory is that Gemini benefits from being able to train on Googles massive internal code base

But does Google actually train its models on its internal codebase? Considering that there’s always the risk of the models leaking proprietary information and security architecture details, I hardly believe they would run that risk.

replies(1): >>44381746 #
kridsdale3 ◴[] No.44381746[source]
Googler here.

We have a second, isolated model that has trained on internal code. The public Gemini AFAIK has never seen that content. The lawyers would explode.

replies(2): >>44381786 #>>44385758 #
1. thimabi ◴[] No.44381786[source]
Oh, you’re right, there are the legal issues as well.

Just out of curiosity, do you see much difference in quality between the isolated model and the public-facing ones?

replies(1): >>44381810 #
2. kridsdale3 ◴[] No.44381810[source]
We actually only got the “2.5” version of the internal one a few days ago so I don’t have an opinion yet.

But when I had to choose between “2.0 with Google internal knowledge” and “2.5 that knows nothing” the latter was always superior.

The bitter lesson indeed.

replies(1): >>44390334 #
3. ◴[] No.44390334[source]