Extending the context length to 1M tokens

Can we all agree that these models far surpass human intelligence now? I mean they process hours worth of audio in less time than it would take a human to even listen. I think the singularity passed and we didn't even notice (which would be expected)

Processing speed is not the metric for measuring intelligence. The same way we have an above average intelligent people taking longer time to think about stuff and coming with better ideas. One can argue that this useful in some aspects but humans have different types of intelligence spectrum that an LLM will lack. Also are you comparing against average person or people on top of their fields or people working in science?

Also human can reason, LLMs currently can't do this in useful way and is very limited by their context in all the trials to make it do that. Not to mention their ability to make new things if they do not exist (and not complete made up stuff that are non-sense) is very limited.

You've hit on the idea that intelligence is not quantifiable by one metric. I completely agree. But you're holding a much different goal for AI than for average people. Modern LLMs are able to produce insights much faster and more accurately than most people (you think you could pass the retrieval tasks in the way that the LLMs do (reading the whole text)?... I really encourage people to try). By that metric (insights/speed), I think they far surpass even the most brilliant. You can claim that that's not intelligence until the cows come home, but any person able to do that would be considered a savant.

I would argue the opposite actually. The same way we don't call someone who are able to do arithmetic calculations very fast a genius if they can't think in more useful mathematical way and construct novel ideas. The samething is happening here, these tools are useful in retrieving and processing current information at high speeds but intelligence is not about the ability to process some data at high speed and then recall them. This is what we actually call servant. It is the ability to build on top this knowledge retrieval and use reason to create new ideas is a closer definition of intelligence and would be better goal.

Let's step back.

1. The vast majority of people never come up with a truly new idea. those that do are considered exceptional and their names go down in history books.

2. Most 'new ideas' are rehashes of old ones.

3. If you set the temperature up on an LLM, it will absolutely come up with new ideas. Expecting an LLM to make a scientific discover a la einstein is ... a bit much, don't you think [1]? When it comes to 'everyday' creativity, such as short poems, songs, recipes, vacation itineraries, etc. ChatGPT is more capable than the vast majority of people. Literally, ask ChatGPT to write you a song about _____, and it will come up with something creative. Ask it for a recipe with ridiculous ingredients and see what it does. It'll make things you've never seen before, generate an image for you and even come up with a neologism if you ask it too. It's insanely creative.

[1] Although I have walked chatgpt through various theoretical physics scenarios and it will create new math for you.

> The vast majority of people never come up with a truly new idea. those that do are considered exceptional and their names go down in history books.

Depends on your definition of "truly" new since any idea could be argued to be a mix of all past ideas. But I see truly new ideas all the time without going down in the history books because most new ideas are incrementally building on what came before or are extremely niche and only a very few turn out to be a massive turning point which has a broad impact which is also only usually evident in retrospect (e.g. blue LEDs was basically trial and error and almost an approach that was given up on, transistors were believed to be impactful but not a huge revolution for computing like they turned out to be, etc etc).