←back to thread

93 points walz | 1 comments | | HN request time: 0.202s | source

I was walking around New York last month during some light rain and noticed about half the people had umbrellas open. When the rain picked up a few minutes later, that number jumped closer to 80%.

It got me thinking it'd be cool to track this somehow, so I built a website! I am taking a sidewalk livestream, feeding it into a YOLO model for people tracking, then sending a frame of each detected person to Gemini 2.0 Flash, which returns structured JSON about each person's clothing and if they're holding an umbrella. I also had fun making the site look like a TV weather channel.

I showed some friends this project and someone mentioned how the legendary Tasks xkcd comic (https://xkcd.com/1425) is out of date now. If you want to check whether a photo has birds in it (or if someone is holding an umbrella), you can just ask an inexpensive vision model for JSON.

1. eber ◴[] No.44366436[source]
I've kinda always wanted this.

I now luckily have a window that looks out into the city and I use what other people are wearing as an indicator for what to wear that day. Definitely helpful on the marginal days where it's maybe shorts, maybe pants, maybe light jacket, maybe sweater-weather. Temperature/wind/humidity tell most of the story, but there's cloud cover, wind direction, morning-to-night temperature swings, etc, that make the decision a bit more iffy.

Cool project! I may need to look into doing something similar.