←back to thread

Video scraping: extracting JSON from a 35s screen capture for 1/10th of a cent

(simonwillison.net)
302 points simonw | 1 comments | 17 Oct 24 12:43 UTC | HN request time: 0.202s | source
Show context
spenczar5 ◴[20 Oct 24 04:08 UTC] No.41892793[source]▶
Suppose I give Gemini a 10 minute video. Will it spend 10 minutes “watching” it if I ask it to extract something? Or does it know how to speed up the video? I assume it must do some sort of preprocessing like extracting keyframes; it surely (?) can’t be looking at the raw encoded video bytes, after all.
replies(1): >>41892809 #
1. simonw ◴[20 Oct 24 04:12 UTC] No.41892809[source]▶
It won't take 10 minutes, but it might still take a minute or two (for Pro) - though Flash and Flash 8B should be significantly faster.

It does process a version of the raw video but it can run that faster than the default video playback rate.

That is quite a bit of detail here: https://ai.google.dev/gemini-api/docs/vision?lang=python#pro...

"The File API service extracts image frames from videos at 1 frame per second (FPS) and audio at 1Kbps, single channel, adding timestamps every second."