I agree with just about all of Matt's points, but I've gotten (and continue to get) so much value out of the people here that I'm not ready to walk.
PG's recent inclusion of "average karma" seems to me a shot over the bow pushing us to recognize the importance of thinking before we speak.
It is shocking to me to see a leader board populated with low karma averages - only 1/3 of the "top" 100 members by total Kamra average over 5 (including Matt).
If there's one piece that we desperately need it's the ability to downvote, tied into average karma which is how I've measured and meter my contribution (and others) to the community.