Llama-OCR: Document to Markdown

I gave it a sentence, which I created by placing 500 circles via a genetic algorithm to form a sentence. And then drew with an actual physical circle:

https://www.instagram.com/marekgibney/p/BiFNyYBhvGr/

Interestingly, it sees the circles just fine, but not the sentence. It replied with this:

    The image contains no text or other elements
    that can be represented in Markdown. It is a
    visual composition of circles and does not
    convey any information that can be translated
    into Markdown format.

I can’t read anything but the „stop“ either without seeing the solution first

I can't read this either.

Edit: at a distance it's easier to read

Why is it interesting? The image does not look like anything, and you need to skew it (by looking at an angle) to see any letters (barely).

If you squint it’s easier too. I wonder if lowering the resolution of the image would make the text visible to ocr.

Based on the fact that squinting works, I applied a Gaussian blur to the image. Here's the response I got:

Markdown:

The provided image is a blurred text that reads "STOP THINKING IN CIRCLES." There are no other visible elements such as headers, footers, subtexts, images, or tables.

Markdown Content:

STOP THINKING IN CIRCLES

As the response is not deterministic, I also tried several times with the unprocessed image but it never worked. However, all the low-pass filter effects I applied worked with a high success rate.

https://imgur.com/q7Zd7fa

I guess blurring it is similar to reducing the resolution or to looking at the image from further away.

It's interesting that the neural net figures out the circles, but not the words. Because the circles are also not so easily apparent from looking closely at the image. It could also be whirly lines.

Was the original LLM ever trained on original material like this?

Pretty cool use of genetic algorithm! Would love to see the code or at least the reward function.

I wonder if you could do a composite image, like bracketed images, and so give the model multiple goes, for which it could amalgamate results. So, you could do an exposure bracket, do a focus/blur, maybe a stretch/compression, or an adjustment for font-height as a proportion of the image.

Feed all of the alternatives to the model, tell it they each have the same textual content?