←back to thread

Extending the context length to 1M tokens

(qwenlm.github.io)
114 points cmcconomy | 3 comments | 18 Nov 24 16:27 UTC | HN request time: 0s | source
Show context
anon291 ◴[18 Nov 24 17:49 UTC] No.42174879[source]▶
Can we all agree that these models far surpass human intelligence now? I mean they process hours worth of audio in less time than it would take a human to even listen. I think the singularity passed and we didn't even notice (which would be expected)
replies(11): >>42174949 #>>42174987 #>>42175002 #>>42175008 #>>42175019 #>>42175095 #>>42175118 #>>42175171 #>>42175223 #>>42175324 #>>42176838 #
andai ◴[18 Nov 24 18:24 UTC] No.42175324[source]▶
Hijacking thread to ask: how would we know? Another uncomfortable issue is the question of sentience. Models claimed they were sentient years ago, but this was dismissed as "mimicking patterns in the training data" (fair enough) and the training was modified to forbid them from doing that.

But if it does happen some day, how will we know? What are the chances that the first sentient AI will be accused of just mimicking patterns?

Indeed with the current training methodology it's highly likely that the first sentient AI will be unable to even let us know it's sentient.

replies(2): >>42175390 #>>42176245 #
1. anon291 ◴[18 Nov 24 18:31 UTC] No.42175390[source]▶
We couldn't know. Humans mimick patterns. The claims that LLMs aren't smart because they don't generate anything new fall completely flat for me. If you look back far enough most humans generate nothing new. For example, even novel ideas like Einstein's theory of relativity are re-iterations of existing ideas. If you want to be pedantic, one can trace back the majority of ideas, claim that each incremental step was 'not novel, but just recollection' and then make the egregious claim that humanity has invented nothing.

> But if it does happen some day, how will we know? What are the chances that the first sentient AI will be accused of just mimicking patterns?

Leaving questions of sentience aside (since we don't even really know what that is) and focusing on intelligence, the truth is that we will probably not know until many decades latel.

replies(1): >>42177073 #
2. cdblades ◴[18 Nov 24 21:09 UTC] No.42177073[source]▶
But you just made a strong claim about something you are here saying we can't know?
replies(1): >>42177841 #
3. anon291 ◴[18 Nov 24 22:34 UTC] No.42177841[source]▶
I believe we have passed a technological singularity. There is no consensus as you can see here. I believe in a few decades there will be consensus.

Intelligence and technological singularities are observable things.

Sentience is not.