←back to thread

Video scraping: extracting JSON from a 35s screen capture for 1/10th of a cent

(simonwillison.net)
302 points simonw | 1 comments | 17 Oct 24 12:43 UTC | HN request time: 0s | source
Show context
pridkett ◴[18 Oct 24 03:48 UTC] No.41876184[source]▶
Video scraping doesn’t need to be just screen captures. I’ve demoed a solution with Gemini where you take a video walking up and down aisles in a retail store and it captured 100% accurate data on product name, quantity/size, sku, and price for a little under 75% of the products. And that was back in January.

This has huge implications for everything from competitive pricing, to understanding store layouts, to creating your own grocery store inflation monitor. Just subtly take a video and process it.

And the models have only gotten better.

replies(4): >>41889606 #>>41891498 #>>41891541 #>>41894068 #
1. TechDebtDevin ◴[19 Oct 24 23:14 UTC] No.41891541[source]▶
I've done this as well with bookshelves at thrift stores that are completely unorganized. I don't want to lose my mind reading every title on every book binding, so I take a picture and ask the model to list all the books on the shelf then I can easily scan through the list to see if something catches my eye.