←back to thread

423 points sohkamyung | 2 comments | | HN request time: 0.423s | source
Show context
visarga ◴[] No.45669657[source]
I recently tried to get Gemini to collect fresh news and show them to me, and instead of using search it hallucinated everything wholesale, titles, abstracts and links. Not just once, multiple times. I am kind of afraid of using Gemini now for anything related to web search.

Here is a sample:

> [1] Google DeepMind and Harvard researchers propose a new method for testing the ‘theory of mind’ of LLMs - Researchers have introduced a novel framework for evaluating the "theory of mind" capabilities in large language models. Rather than relying on traditional false-belief tasks, this new method assesses an LLM’s ability to infer the mental states of other agents (including other LLMs) within complex social scenarios. It provides a more nuanced benchmark for understanding if these systems are merely mimicking theory of mind through pattern recognition or developing a more robust, generalizable model of other minds. This directly provides material for the construct_metaphysics position by offering a new empirical tool to stress-test the computational foundations of consciousness-related phenomena.

> https://venturebeat.com/ai/google-deepmind-and-harvard-resea...

The link does not work, the title is not found in Google Search either.

replies(8): >>45669725 #>>45670064 #>>45670405 #>>45670834 #>>45671889 #>>45673663 #>>45676497 #>>45678588 #
1. mckngbrd ◴[] No.45671889[source]
What version of Gemini were you using? i.e. were you calling it locally via the API or thru their Gemini or AI Studio web apps?

Not every LLM app has access to web / news search capabilities turned on by default. This makes a huge difference in what kind of results you should expect. Of course, the AI should be aware that it doesn't have access to web / news search, and it should tell you as much rather than hallucinating fake links. If access to web search was turned on, and it still didn't properly search the web for you, that's a problem as well.

replies(1): >>45672458 #
2. visarga ◴[] No.45672458[source]
Gemini 2.5 Pro and it was this month, so probably the latest version.