I used this in earnest yesterday on my Zillow saved listings. I prompted it to analyze the listings (I've got about 70 or so saved) and summarize the most recent price drops for each one and it mostly failed at the task. It gave the impression that it paginated through all the listings, but I don't think it actually did. I think the mechanism by which it works, which is to click links and take screenshots and analyze them must be some kind of token efficiency trade-off (as opposed to consuming the DOM) and it seems not great at the task.
As a reformed AI skeptic I see the promise in a tool like this, but this is light years behind other Anthropic products in terms of efficacy. Will be interesting to see how it plays out though.
replies(5):