←back to thread

Fei-Fei Li: Spatial intelligence is the next frontier in AI [video]

(www.youtube.com)
289 points sandslash | 2 comments | 01 Jul 25 14:00 UTC | HN request time: 0.002s | source
Show context
sabman ◴[03 Jul 25 07:30 UTC] No.44452538[source]▶
We've been working on this challenge in the satellite domain with https://earthgpt.app. It’s a subset of what Fei-Fei is describing, but comes with its own unique issues like handling multi-resolution sensors and imagery with hundreds of spectral bands. Think of it as computer vision, but in n-dimensions.

Happy to answer questions if you're curious. PS. still in early beta, so please be gentle!

replies(2): >>44452795 #>>44453375 #
fnands ◴[03 Jul 25 08:14 UTC] No.44452795[source]▶
Hey, cool project!

Do you actually pass the images to the model, or just the metadata/stats?

replies(1): >>44453732 #
sabman ◴[03 Jul 25 10:57 UTC] No.44453732[source]▶
Thanks! This live demo uses metadata and stats only. Right now we are testing ViTs and Foundation Models as well. But quality of results from EO FMs haven't been worth the inference cost so far. Early days though. Also starting to fine tune models for specific downstream tasks ourselves.
replies(1): >>44453874 #
1. fnands ◴[03 Jul 25 11:24 UTC] No.44453874{3}[source]▶
Cool, makes sense.

Yeah, have you considered maybe looking into just running it on embeddings [1], instead of the imagery itself? Would save on most of the inference cost, at the cost of flexibility (i.e. you are locked into whatever embeddings have been created).

[1] https://developers.google.com/earth-engine/datasets/catalog/...

replies(1): >>44462164 #
2. sabman ◴[04 Jul 25 07:46 UTC] No.44462164[source]▶
ah yes we have been testing other embedding models but not google's. I'll try this too. Its interesting most of them are doing land cover classes which is kinda solved already. We are also testing mixing agenic workflows with smaller directed prompts for users to provide the classes. Incidentally we are Berlin based. We should grab a coffee :)