←back to thread

Coming Soon to Hacker News: Pending Comments

661 points pg | 8 comments | 21 Mar 14 23:49 UTC | HN request time: 0.354s | source | bottom

A surprisingly long time ago (2013 was a busy year) I mentioned a new plan to improve the quality of comments on Hacker News:

https://news.ycombinator.com/item?id=6009523

Since I'm going to check out of HN at the end of this YC cycle, this was my last chance to get this done. I didn't want the people who are going to inherit HN from me to have to build it as their first project, because it interacts with so many different bits of the code in such subtle ways.

So I found time to implement pending comments this past week, and with any luck it will launch tonight. Since it's a big change, I wanted to warn HN users in advance.

Here's how it currently works. From now on, when you post a comment, it won't initially be live. It will be in a new state called pending. Comments get from pending to live by being endorsed by multiple HN users with over 1000 karma. Those users will see pending comments, and will be able to endorse them by clicking on an "endorse" link next to the "flag" link.

Someone who has a pending comment will have to wait till it goes live to post another. We're hoping that good comments will get endorsed so quickly that there won't be a noticeable delay.

You can currently beat the system by posting an innocuous comment, waiting for it to be endorsed, and then after it's live, changing it to say something worse. We explicitly ask people not to do this. While we have no software for catching it, humans will notice, and we'll ban you.

Along with the change in software will come a change in policy. We're going to ask users with the ability to endorse comments only to endorse those that:

1. Say something substantial. E.g. not just a throwaway remark, or the kind of "Yes you did, No I didn't" bickering that races toward the right side of the page and no one cares about except the participants.

2. Say it without gratuitous nastiness. In particular, a comment in reply to another comment should be written in the spirit of colleagues cooperating in good faith to figure out the truth about something, not politicians trying to ridicule and misrepresent the other side.

People who regularly endorse comments that fail one or both of these tests will lose the ability to endorse comments. So if you're not sure whether you should endorse a comment, don't. There are a lot of people on HN. If a point is important, someone else will probably come along and make it without gratuitous nastiness.

I hope this will improve the quality of HN comments significantly, but we'll need your help to make it work, and your forbearance if, as usually happens, some things go wrong initially.

1. bsamuels ◴[22 Mar 14 00:07 UTC] No.7445902[source]▶
I think this change makes the mistake that people with lots of karma are good contributors.

I think it should be based on weighted karma/comment in addition to total karma. Imagine two users - one with 1500 karma & 1.1 karma/comment versus one with 600 karma & 10 karma/comment, which one would you trust to be able to judge what is a good comment and what isn't?

The total karma weight is to give at least some favor to frequent users, but not too much.

In addition, if someone replies to your comment, you should be able to at least see their comment regardless of your karma or whether it's been greenlit.

replies(5): >>7445967 #>>7445976 #>>7445990 #>>7446117 #>>7448676 #
2. tedsanders ◴[22 Mar 14 00:14 UTC] No.7445967[source]▶
That's a good point, but I think it would have an unintended negative consequence of punishing late comments, which tend to have low karma but may nonetheless be worthwhile.

It would also punish comments on 'new' threads, which often stagnate with 0 or 1 upvotes.

3. dudus ◴[22 Mar 14 00:15 UTC] No.7445976[source]▶
Karma/Comment is not a very good metric as well. As someone that posted a big post that stayed in the Front Page for long can amass a lot of karma in little time. Giving them a very high Karma/Comment. You would have to remove the outliers.
replies(2): >>7445997 #>>7446167 #
4. tokenadult ◴[22 Mar 14 00:16 UTC] No.7445990[source]▶
I think this change makes the mistake that people with lots of karma are good contributors.

That's not an invariant property of participants with karma over 1000 points, to be sure, but the population of participants with over 1000 points skews to include a fair number of people who have been long-term contributors (patio11, who has my same join date and deservedly much more karma, immediately comes to mind among several other participants). What's practical about this bright-line rule is that it strictly limits the number of participants who can review pending comments at all, and allows the moderation team here to go to the next step of evaluating the reviewers, while ALL OF US are put on notice that what's desired here is substantive, polite comments.

So while I agree with you that the relationship "high karma implies good contributor" is not an invariantly true relationship, it has enough heuristic value to start as a seed for the system as further hand-tweaking of review power by the moderation team responds to your legitimate concern here.

5. jamesaguilar ◴[22 Mar 14 00:17 UTC] No.7445997[source]▶
A perfect opportunity to use comment-karma H index! http://en.wikipedia.org/wiki/H-index
6. ilyagr ◴[22 Mar 14 00:32 UTC] No.7446117[source]▶
I think that bsamuels' point about replies is also important: I sometimes reply to comments with the intention to be helpful to the author of the comment, not necessarily everybody else.
7. corresation ◴[22 Mar 14 00:39 UTC] No.7446167[source]▶
Karma/Comment is a poor metric, I think, because it encourages the strike attack comment style.

If I post something and people post counterpoints or questions or concerns, I will respond. But the truth is that from a pure karma average perspective, by far the best action would be to simply leave the threads hanging, the single post floating at the top of karma heaven. That works great from a pure karma perspective, but it subdues conversation because every point is a single attack.

It also encourages the stealthy comment editor. While Paul mentions this, it is something that happens regularly -- people make a comment, someone responds unfavourably, and the original person subtly edits their comment to make them look unfairly attacked. This is a specific problem on HN given that quoting is generally discouraged. So everyone piles on the downvotes and upvotes, respectively, to right this seeming injustice.

Karma on HN is easily gamed, as it is elsewhere. It is unfortunate when we care too much about it because it leads to completely artificial conversations.

8. thaumasiotes ◴[22 Mar 14 11:44 UTC] No.7448676[source]▶
> I think this change makes the mistake that people with lots of karma are good contributors.

You're making the related mistake that comments with lots of karma are good comments.

By coincidence, I (at the time of writing) have roughly 1500 karma at 2.2 / comment. Here's a selection of some of my sub-10-karma comments that I happen to like:

https://news.ycombinator.com/item?id=7203273 (for reasons I don't understand, 0 karma)

https://news.ycombinator.com/item?id=7276538 (2)

https://news.ycombinator.com/item?id=7156141 (3)

https://news.ycombinator.com/item?id=7396659 (4)

https://news.ycombinator.com/item?id=7239733 (4)

https://news.ycombinator.com/item?id=7352327 (5)

Here's every 10+ karma comment I've made in the past 50 days:

https://news.ycombinator.com/item?id=7206460 (10, with a 12-point comment two levels down)

https://news.ycombinator.com/item?id=7329150 (12)

https://news.ycombinator.com/item?id=7296981 (11)

https://news.ycombinator.com/item?id=7206436 (14)

https://news.ycombinator.com/item?id=7194772 (22)

Weirdly, the intuition I got from going back over these is that I'd probably use a threshold of 8 to divide "popular" comments from "unpopular" ones.

Anyway, I see a clear trend in the 10+ comments: they either openly mock someone, or they take a stand on a contentious political issue (the 22-pointer does both!). My favorite of them all is the 5-pointer, and I dearly hoped that someone would reply to it... but no.

Obviously, I've got plenty of less-worthwhile 1- and 2-point comments to my name. But I'm not at all convinced that someone with 10 average karma per comment is making better comments than someone with an average of two; it seems more likely that they're either demagoguing or just witty, possibly nastily witty.

postscript: average karma of some bright lights:

patio11: 15.2

cperciva: 11.3

tokenadult: 6.1

tptacek: 5.4