←back to thread

423 points sohkamyung | 2 comments | | HN request time: 0.001s | source
Show context
visarga ◴[] No.45669657[source]
I recently tried to get Gemini to collect fresh news and show them to me, and instead of using search it hallucinated everything wholesale, titles, abstracts and links. Not just once, multiple times. I am kind of afraid of using Gemini now for anything related to web search.

Here is a sample:

> [1] Google DeepMind and Harvard researchers propose a new method for testing the ‘theory of mind’ of LLMs - Researchers have introduced a novel framework for evaluating the "theory of mind" capabilities in large language models. Rather than relying on traditional false-belief tasks, this new method assesses an LLM’s ability to infer the mental states of other agents (including other LLMs) within complex social scenarios. It provides a more nuanced benchmark for understanding if these systems are merely mimicking theory of mind through pattern recognition or developing a more robust, generalizable model of other minds. This directly provides material for the construct_metaphysics position by offering a new empirical tool to stress-test the computational foundations of consciousness-related phenomena.

> https://venturebeat.com/ai/google-deepmind-and-harvard-resea...

The link does not work, the title is not found in Google Search either.

replies(8): >>45669725 #>>45670064 #>>45670405 #>>45670834 #>>45671889 #>>45673663 #>>45676497 #>>45678588 #
HWR_14 ◴[] No.45670834[source]
Why would you want Gemini to do this instead of just going to a news site (or several news sites) and reading what the headlines they wrote?
replies(2): >>45672488 #>>45674954 #
visarga ◴[] No.45672488[source]
I wanted to use the agentic powers of the model to dig for specific kinds of news, and use iterative search as well. I think when LLMs use tools correctly this kind of search is more powerful than simple web search. It also has better semantic capabilities, so in a way I wanted to make my own LLM powered news feed.
replies(2): >>45673296 #>>45674800 #
1. SrslyJosh ◴[] No.45674800[source]
> I wanted to use the agentic powers of the model

Do you have an in-depth understanding of how those "agentic powers" are implemented? If not, you should probably research it yourself. Understanding what's underneath the buzzwords will save you some disappointment in the future.

replies(1): >>45677345 #
2. visarga ◴[] No.45677345[source]
I think I do, I have been in ML for 12 years and followed transformers since their invention. Also been using LLM daily since they appeared, personally.