Video scraping: extracting JSON from a 35s screen capture for 1/10th of a cent

Video scraping doesn’t need to be just screen captures. I’ve demoed a solution with Gemini where you take a video walking up and down aisles in a retail store and it captured 100% accurate data on product name, quantity/size, sku, and price for a little under 75% of the products. And that was back in January.

This has huge implications for everything from competitive pricing, to understanding store layouts, to creating your own grocery store inflation monitor. Just subtly take a video and process it.

And the models have only gotten better.

> This has huge implications for everything from competitive pricing, to understanding store layouts

Even smaller stores have been monitoring their competitors since a long time.

> your own grocery store inflation monitor

You could also check your itemized bill.

Holy shit I’ve been thinking about doing this for my local grocery stores, so I can stop wondering things like “who has lady fingers”.

I was wondering how well it would work. That’s such good news

I've done this as well with bookshelves at thrift stores that are completely unorganized. I don't want to lose my mind reading every title on every book binding, so I take a picture and ask the model to list all the books on the shelf then I can easily scan through the list to see if something catches my eye.

would be way easier if stores interface with search engines directly. majority of stores have inventory systems. just a matter of time until they intehrate. if you search for a product you can sometimes see places that have it nearby.

> You could also check your itemized bill.

But only for the things you buy.

Note too, some big retail stores actually have a "license" or "contract" for customers hiding behind the service desk, and often, video recording is one of the things they forbid. It's not "illegal" to do so, but if they catch you, and insist you leave, and you refuse, now you're trespassing, and that has legal consequences.

I've explored the mystery shopper apps a few times.

A common task is to take photos of shelf's and the products / pricing.

Its framed as "make sure our employees are doing it correct" but based on the strict image requirements (needed for later computer processing) I have the feeling it is actually a competitor trying to get shelf and price info. It feels a little cloak and dagger.

Is there any possibility to view such demo? Or learning something more?

Often their own data is wrong.

This is an old thread and will probably be lost now, but this sort of tool has been industrialized now. At the library book sales in my area, you'll see a handful of people there right as they open, they'll have either a scanner device or an app on their phone. They just scan over the spines of the books and it beeps when it finds something interesting. It highlights the book and they pick it out.

On the one hand, great application of technology. On the other hand, the folks using them have zero interest in reading the books. So, when the library happens to have a nearly brand new copy of "Harry Potter and the Order of the Phoenix: Illustrated Edition" there's almost no chance that someone in the community who wants to read the book will get it. Instead it gets snapped up by the resellers who make a couple of bucks off sending it to someone else who may be across the country.