I dig what you're saying...
I have to quote one of my favorite thinkers here:
"Society does not consist of individuals but expresses the sum of interrelations, the relations within which these individuals stand." - K. Marx
This world knowledge is built upon piece by piece, the conceptual tools of the past create the conceptual tools of the future, that line is drawn through books and projected through minds, again onto books. This whole society depends deeply on cohesion and cultural continuation.
Our intellectual thread is the cultural knowledge and technological progress itself, its not even down to great individuals alone. I think believing in great individuals is a product of a sort of personality-fetishism (though individuals can do great things, if that makes sense).
This fetishism or mystification of the person who contributes I view as a product of an old frame of thinking which is called philosophical liberalism. This framework does this because it posits that all peoples exist under equal social value (political, legal and economical), thus people who contribute more must have a greater capacity that is innate and unexplainable or untraceable; inherent. Its a widespread philosophical frame of thought that does not consider the conditions of the individual.
We most see this employed with rich people. We hear they are truly great, savvy, exceptional individuals, when in reality a lot of the times the explanation for the vast majority of the rich is that they had rich parents. Where would you be if your parents owned an emerald mine? or Where would you be if your parents gave you a small loan of a million dollars?
In the same vein this human progress that we encounter, which seems to be carried on the backs of the Newtons and Einsteins of the world, is in fact a steady drip-feed of collective human knowledge that gets compiled and analyzed, made consistent and expounded upon by a few persons every certain amount of time. No lesser of a feat, mind you, the work is still there. I am not minimizing these persons, but contextualizing them.
[Insert the "on shoulders of giants" quote here]. Is a great example of humility and awareness by a visionary.
One thing I find impressive at times is the vast amount of German intellectuals throughout history, which upon looking at history can be explained by their colonial exploits leading to greater national wealth, leisure, and cultural amplification. This is often the case with Europe and the USA as well.
So there is a chance that we are all base-level intelligence, since we are all essentially the same species. What changes that is access to the cultural wealth of information, and not only access to this cultural wealth of information but a CULTURE OF ACCESS to that wealth of info. A level of social development around you that enables you.
People would rather immediately jump to physiological and even genetic explanations of intelligence rather than look at the social context of the individuals involved. This is because of the flaws of philosophical liberalism at contextualizing and actually scientifically looking at the world around us.
Again: there's a good chance that we are all just base level intelligence. What we know is actually different between us is the preparation and economic/social context of the individuals.