Video scraping: extracting JSON from a 35s screen capture for 1/10th of a cent

Video scraping doesn’t need to be just screen captures. I’ve demoed a solution with Gemini where you take a video walking up and down aisles in a retail store and it captured 100% accurate data on product name, quantity/size, sku, and price for a little under 75% of the products. And that was back in January.

This has huge implications for everything from competitive pricing, to understanding store layouts, to creating your own grocery store inflation monitor. Just subtly take a video and process it.

And the models have only gotten better.

> This has huge implications for everything from competitive pricing, to understanding store layouts

Even smaller stores have been monitoring their competitors since a long time.

> your own grocery store inflation monitor

You could also check your itemized bill.

Holy shit I’ve been thinking about doing this for my local grocery stores, so I can stop wondering things like “who has lady fingers”.

I was wondering how well it would work. That’s such good news

I've done this as well with bookshelves at thrift stores that are completely unorganized. I don't want to lose my mind reading every title on every book binding, so I take a picture and ask the model to list all the books on the shelf then I can easily scan through the list to see if something catches my eye.

would be way easier if stores interface with search engines directly. majority of stores have inventory systems. just a matter of time until they intehrate. if you search for a product you can sometimes see places that have it nearby.

> You could also check your itemized bill.

But only for the things you buy.

Note too, some big retail stores actually have a "license" or "contract" for customers hiding behind the service desk, and often, video recording is one of the things they forbid. It's not "illegal" to do so, but if they catch you, and insist you leave, and you refuse, now you're trespassing, and that has legal consequences.

I've explored the mystery shopper apps a few times.

A common task is to take photos of shelf's and the products / pricing.

Its framed as "make sure our employees are doing it correct" but based on the strict image requirements (needed for later computer processing) I have the feeling it is actually a competitor trying to get shelf and price info. It feels a little cloak and dagger.

Is there any possibility to view such demo? Or learning something more?

Often their own data is wrong.