←back to thread

Video scraping: extracting JSON from a 35s screen capture for 1/10th of a cent

(simonwillison.net)
302 points simonw | 3 comments | 17 Oct 24 12:43 UTC | HN request time: 0.706s | source
Show context
pridkett ◴[18 Oct 24 03:48 UTC] No.41876184[source]▶
Video scraping doesn’t need to be just screen captures. I’ve demoed a solution with Gemini where you take a video walking up and down aisles in a retail store and it captured 100% accurate data on product name, quantity/size, sku, and price for a little under 75% of the products. And that was back in January.

This has huge implications for everything from competitive pricing, to understanding store layouts, to creating your own grocery store inflation monitor. Just subtly take a video and process it.

And the models have only gotten better.

replies(4): >>41889606 #>>41891498 #>>41891541 #>>41894068 #
1. tgv ◴[19 Oct 24 18:29 UTC] No.41889606[source]▶
> This has huge implications for everything from competitive pricing, to understanding store layouts

Even smaller stores have been monitoring their competitors since a long time.

> your own grocery store inflation monitor

You could also check your itemized bill.

replies(1): >>41892549 #
2. jasonjayr ◴[20 Oct 24 02:58 UTC] No.41892549[source]▶
> You could also check your itemized bill.

But only for the things you buy.

Note too, some big retail stores actually have a "license" or "contract" for customers hiding behind the service desk, and often, video recording is one of the things they forbid. It's not "illegal" to do so, but if they catch you, and insist you leave, and you refuse, now you're trespassing, and that has legal consequences.

replies(1): >>41893248 #
3. pests ◴[20 Oct 24 06:07 UTC] No.41893248[source]▶
I've explored the mystery shopper apps a few times.

A common task is to take photos of shelf's and the products / pricing.

Its framed as "make sure our employees are doing it correct" but based on the strict image requirements (needed for later computer processing) I have the feeling it is actually a competitor trying to get shelf and price info. It feels a little cloak and dagger.