Gemini 2.5 Flash Image

Like most image generators, it didn’t pass the piano keyboard test. (Black keys are wrong.)

https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%...

Interesting! I feel like that's maybe similar to the business of being able to correctly generate images of text— it looks like the idea of a keyboard to a non-musician, but is immediately wrong to someone who is actually familiar with it at all.

I wonder if the bot is forced to generate something new— certainly for a prompt like that it would be acceptable to just pick the first result off a google image search and be like "there, there's your picture of a piano keyboard".

What is the piano keyboard test? Your link requires granting AI Studio access to Google Drive, which I do not want to do.

Anything that is heavily periodic can definitely trip up image gen - that being I just used Flux Kontext T2I and got a got pretty close (disregard the hammers though since thats a right mess). Only towards the upper register did it start to make mistakes.

https://imgur.com/a/fyX42my

Doesn't pass the analog clock test either.

Just ask it to generate a correct piano keyboard. It's something the current gen of image generator AIs fail at.

The selling point of this model really seems to be it's consistency between generations rather than it's raw generating ability.

for instance:

https://aistudio.google.com/app/prompts/1gTG-D92MyzSKaKUeBu2...

or my "hands with palms facing down" test.... no matter how hard I try it just can't get open hands, palms down.

Like most image models, except GPT-4o, it also didn't pass the wooden Penrose triangle test. (It creates normal triangles.)

Are their models that have vector space that includes ideas, not just words/media but not entirely corporeal aspects?

So when generating a video of someone playing a keyboard the model would incorporate the idea of repeating groups of 8 tones, which is a fixed ideational aspect which might not be strongly represented in words adjacent to "piano".

It seems like models need help with knowing what should be static, or homomorphic, across or within images associated with the same word vectors and that words alone don't provide a strong enough basis [*1] for this.

*1 - it's so hard to find non-conflicting words, obviously I don't mean basis as in basis vectors, though there is some weak analogy.

I guess the vast majority of images have the palms the other way, that this biases the output. It's like how we misinterpret images to generate optical illusions, because we're expecting valid 3D structures (Escher's staircases, say).

It's probably just a matter of rerolling a few times. I was able to get it around 25% of the time.

https://imgur.com/a/H9gH3Zy

Yes - it's the same reason generating a 5-leaf clover fails - massive amounts of training data that predisposes the model against it.

How would you encode those ideas?

Do most humans pass?

I can’t see it. You probably need to set permissions to “anyone with the link can access.”

2-2-1-2-2-2-1

Presumably most humans with a camera do

I still feel like most humans would fail, haha.

Most humans fail at 4 digits multiplication, or drawing a cube in perspective.

Failed my horizontal text test as well.

Maybe, but anyone who knows what a chromatic scale is should be able to reason it out. E# == F, B# == C, so no black keys between those.

that's pretty good. I was using a cartoon girl as an example of a dance move for kids.

https://g.co/gemini/share/0e0de0d42029

I don't know, in part that's why I asked ... I wonder if there's a way to provide a loosely-defined space.

Perhaps it's a second word-vector space that allows context defined associations? Maybe it just needs tighter association of piano_keyboard with 8-step_repetition??