Gemini CLI

Been using Claude Code (4 Opus) fairly successfully in a large Rust codebase, but sometimes frustrated by it with complex tasks. Tried Gemini CLI today (easy to get working, which was nice) and it was pretty much a failure. It did a notably worse job than Claude at having the Rust code modifications compile successfully.

However, Gemini at one point output what will probably be the highlight of my day:

"I have made a complete mess of the code. I will now revert all changes I have made to the codebase and start over."

What great self-awareness and willingness to scrap the work! :)

Personally my theory is that Gemini benefits from being able to train on Googles massive internal code base and because Rust has been very low on uptake internally at Google, especially since they have some really nice C++ tooling, Gemini is comparatively bad at Rust.

Gemini has some fun failure modes. It gets "frustrated" when changes it makes doesn't work, and replies with oddly human phrases like "Well, that was unexpected" and then happily declares that (I see the issue!) "the final tests will pass" when it's going down a blind alley. It's extremely overconfident by default and much more exclamatory without changing the system prompt. Maybe in training it was taught/figured out that manifesting produces better results?

That's interesting. I've tried Gemini 2.5 Pro from time to time because of the rave reviews I've seen, on C# + Unity code, and I've always been disappointed (compared to ChatGPT o3 and o4-high-mini and even Grok). This would support that theory.

It also gets really down on itself, which is pretty funny (and a little scary). Aside from the number of people who've posted online about it wanting to uninstall itself after being filled with shame, I had it get confused on some Node module resolution stuff yesterday and it told me it was deeply sorry for wasting my time and that I didn't deserve to have such a useless assistant.

Out of curiosity, I told it that I was proud of it for trying and it had a burst of energy again and tried a few more (failing) solution, before going back to it's shameful state.

Then I just took care of the issue myself.

So far I've found Gemini CLI is very good at explaining what existing code does.

I can't say much about writing new code though.

After a particular successful Claude Code task I praised it and told it to "let's fucking go!" to which it replied that loved the energy and proceeded to only output energetic caps lock with fire emojis. I know it's all smoke and mirrors (most likely), but I still get a chuckle out of this stuff.

Interesting, Gemini must be a monster when it comes to Go code then. I gotta try it for that

Claude will do the same start over if things get too bad. At least I've seen it when its edits went haywire and trashed everything.

Same here. Tried to implement a new feature on one of our apps to test it. It completely screwed things up. Used undefined functions and stuff. After a couple of iterations of error reporting and fixing I gave up.

Claude did it fine but I was not happy with the code. What Gemini came up with was much better but it could not tie things together at the end.

There is way more Java and C++ than Go at Google.

> Personally my theory is that Gemini benefits from being able to train on Googles massive internal code base

But does Google actually train its models on its internal codebase? Considering that there’s always the risk of the models leaking proprietary information and security architecture details, I hardly believe they would run that risk.

Googler here.

We have a second, isolated model that has trained on internal code. The public Gemini AFAIK has never seen that content. The lawyers would explode.

I tried it too, it was so bad. I got the same "revert" behaviour after only 15 minutes.

Oh, you’re right, there are the legal issues as well.

Just out of curiosity, do you see much difference in quality between the isolated model and the public-facing ones?

We actually only got the “2.5” version of the internal one a few days ago so I don’t have an opinion yet.

But when I had to choose between “2.0 with Google internal knowledge” and “2.5 that knows nothing” the latter was always superior.

The bitter lesson indeed.

Sounds like you can use gemini to create the initial code, then have claude review and finalise what gemini comes up with

>Personally my theory is that Gemini benefits from being able to train on Googles massive internal code base and because Rust has been very low on uptake internally at Google, especially since they have some really nice C++ tooling, Gemini is comparatively bad at Rust.

Were they to train it on their C++ codebase, it would not be effective on account of the fact that they don't use boost or cmake or any major stuff that C++ in the wider world use. It would also suggest that the user make use of all kinds of non-available C++ libraries. So no, they are not training on their own C++ corpus nor would it be particularly useful.

I asked it to do a comparatively pedestrian task: write a script to show top 5 google searches.

First it did the search itself and then added "echo" for each of them - cute

Then it tried to use pytrends which didn't go anywhere

Then it tried some other paid service which also didn't go anywhere

Then it tried some other stuff which also didn't go anywhere

Finally it gave up and declared failure.

It will probably be useful as it can do the modify/run loop itself with all the power of Gemini but so far, underwhelming.

Tangental, but I worry that LLMs will cause a great stagnation in programming language evolution, and possibly a bunch of tech.

I've tried using a few new languages and the LLMs would all swap the code for syntactically similar languages, even after telling them to read the doc pages.

Whether that's for better or worse I don't know, but it does feel like new languages are genuinely solving hard problems as their raison d'etre.

As go feels like a straight-jacket compared to many other popular languages, it’s probably very suitable for an LLM in general.

Thinking about it - was this not the idea of go from the start? Nothing fancy to keep non-rocket scientist away from foot-guns, and have everyone produce code that everyone else can understand.

Diving in to a go project you almost always know what to expect, which is a great thing for a business.

Excuse me why was this downvoted so aggressively??

Not just that, I think this will happen on multiple levels too. Think de-facto ossified libraries, tools, etc.

LLMs thrive because they had a wealth of high-quality corpus in the form os Stack Overflow, Github, etc. and ironically their uptake is causing a strangulation of that source of training data.

What model do your lawyers run on?

This was also my exact experience. I was pretty excited because I usually use Gemini Pro 2.5 when Claude Code gets stuck by pasting the whole code and asking questions and it was able to get me out of a few pickles a couple of times.

Unfortunately the CLI version wasn't able to create coherent code or fix some issues I had in my Rust codebase as well.

Here's hope that it eventually becomes great.

How can they train on internal codebase without leaking specifics?