It's nepotism and it's enshrined in the recommendation letter system.
The fact that GRE is a joke and that in the USA undergrad, even at the elite level, is usually just a glorified partying program doesn't help either.
Its basically a filter on who can stick with a program for an extended period of time. You know, skills you need if you going to do a 5-7 year PhD.
It certainly doesn't prepare you for research, unless you specifically seek out certain opportunities.
Although ironically, it turns out the 4.0 GPA types may not make the best researchers - at least in experimental science. Better GPA make better grad students up to a certain point - then after a certain point (around 3.5-3.6) there is a negative correlation (except medical school where that doesn't happen). I've heard this from multiple professors.