←back to thread

Coming Soon to Hacker News: Pending Comments

661 points pg | 4 comments | 21 Mar 14 23:49 UTC | HN request time: 0.002s | source

A surprisingly long time ago (2013 was a busy year) I mentioned a new plan to improve the quality of comments on Hacker News:

https://news.ycombinator.com/item?id=6009523

Since I'm going to check out of HN at the end of this YC cycle, this was my last chance to get this done. I didn't want the people who are going to inherit HN from me to have to build it as their first project, because it interacts with so many different bits of the code in such subtle ways.

So I found time to implement pending comments this past week, and with any luck it will launch tonight. Since it's a big change, I wanted to warn HN users in advance.

Here's how it currently works. From now on, when you post a comment, it won't initially be live. It will be in a new state called pending. Comments get from pending to live by being endorsed by multiple HN users with over 1000 karma. Those users will see pending comments, and will be able to endorse them by clicking on an "endorse" link next to the "flag" link.

Someone who has a pending comment will have to wait till it goes live to post another. We're hoping that good comments will get endorsed so quickly that there won't be a noticeable delay.

You can currently beat the system by posting an innocuous comment, waiting for it to be endorsed, and then after it's live, changing it to say something worse. We explicitly ask people not to do this. While we have no software for catching it, humans will notice, and we'll ban you.

Along with the change in software will come a change in policy. We're going to ask users with the ability to endorse comments only to endorse those that:

1. Say something substantial. E.g. not just a throwaway remark, or the kind of "Yes you did, No I didn't" bickering that races toward the right side of the page and no one cares about except the participants.

2. Say it without gratuitous nastiness. In particular, a comment in reply to another comment should be written in the spirit of colleagues cooperating in good faith to figure out the truth about something, not politicians trying to ridicule and misrepresent the other side.

People who regularly endorse comments that fail one or both of these tests will lose the ability to endorse comments. So if you're not sure whether you should endorse a comment, don't. There are a lot of people on HN. If a point is important, someone else will probably come along and make it without gratuitous nastiness.

I hope this will improve the quality of HN comments significantly, but we'll need your help to make it work, and your forbearance if, as usually happens, some things go wrong initially.

Show context
beloch ◴[22 Mar 14 01:15 UTC] No.7446449[source]▶
My may concern with this system: Sledgehammer meets tack.

The comments on HN aren't perfect, but they're far from bad when compared to other sites of this nature. There has been a downwards trend most probably due to the increasing popularity of HN. A response is warranted. However, this system has the potential to silence a lot of high quality comments on any threads that aren't on the front-page for an extended period of time. Thus, you get a feedback loop. Good posts require quality discussion to stay on top, but must stay on top to get quality discussion going with this added approval lag.

I think you should ease these changes in as conservatively and gradually as possible. For example, apply it only to the top page at first, and reduce the number of endorsements required for display to 1. You might also consider merely greying out comments that have not yet been endorsed, as currently happens to down-voted comments. Another option would be to apply the endorsement system only after threads have reached a certain age so as to jump-start discussions. Additionally, I would recommend that authors of a parent post should be able to see all child posts regardless of their karma. Below, Babuskov raised the point that the endorsement system will obstruct useful back-and-forth discussions between sub-kilokarma users in buried threads that often takes the place of a private messaging system on HN. This would fix that more effectively than merely reducing the endorsement requirement.

You should not entertain any illusions that you can flip the switch and watch this system work perfectly, and that you will therefore be able to avoid confusing people with many changes over a lengthy period of time. Tweaking will almost certainly be required.

replies(13): >>7446750 #>>7446782 #>>7446797 #>>7446806 #>>7446997 #>>7447135 #>>7447144 #>>7447370 #>>7447425 #>>7447580 #>>7449121 #>>7449627 #>>7450193 #
doorhammer ◴[22 Mar 14 02:03 UTC] No.7446782[source]▶
To me a problem with karma systems that are based on a specific relatively well-defined ruleset is that you still end up with a score that doesn't really convey much. For instance, the karma I have is made up of meaningful discussions, which involve telling others about relevant facts, as well as one liner jokes and a few quotes from random dead folks that are smarter than I am.

I always wonder how useful it is to really try to get down to defining what it is you want out of the conversations, and then giving the ability to narrow based on that. Obviously, the narrowing would have to be quick, because it's hard enough to get people to click an up/down arrow once, but if it were to include even two or three more specific categories, or flags, could you put enough information into the system to automate some of the problem folks.

For instance, having a way for users to flag a post as vitriolic, or not containing real information, and then collating the data and either stopping that user from posting, or putting them into a state where their posts are pending.

I also wanted to mention that it seems as though he does address not having illusions in the last sentence: "I hope this will improve the quality of HN comments significantly, but we'll need your help to make it work, and your forbearance if, as usually happens, some things go wrong initially."

I think all of that spitballing is predicated on having a well-defined course you'd like conversations to follow with well-defined rules that don't block the flow of useful discourse, or stifles the community.

This isn't a topic I've done any research on. Just throwing some thoughts out there. I appreciate the parent post, though, because it raised a lot of notions I hadn't thought of immediately when I read the pending idea. I think I tend to lean toward giving people with higher karma the ability to flag people into a status that makes there posts go into pending, according to a set of defined criteria (like were posted) rather than starting folks in that state. I think it would be less impactful to the rate of conversation, but obviously then you have people who make accounts to get around it, which would be solved by the solution to be implemented.

Also everything I'm suggesting is complicated and possibly overkill/overengineered. I wonder if there's an elegant way to get more useful classification information into the voting system to give it more focussed goals.

Well. Enough rambling on my part.

replies(2): >>7447742 #>>7448497 #
commandar ◴[22 Mar 14 10:10 UTC] No.7448497[source]▶
It's been a decade since I've posted there with any regularity, but this more or less lays out why I feel like I still haven't seen a more elegant solution to moderating comments than Slashdot's metamoderation system.

Comments not only receive a +/- minus score, but they receive a reason for that score. Additionally, meta-moderators are selected randomly from eligible moderators who go behind and moderate whether the scores given to a particular comment were justified. Get metamoderated down often enough and you lose your ability to moderate.

It baffles me that every site on the web keeps using simple up/down systems and then complains about how hard it is to generate quality discussion when this is hardly a new problem set.

replies(2): >>7448618 #>>7448886 #
1. philh ◴[22 Mar 14 11:21 UTC] No.7448618{3}[source]▶
Do you feel like slashdot has high quality discussions? I don't. The problem isn't new, but it also isn't solved.
replies(2): >>7448692 #>>7449575 #
2. commandar ◴[22 Mar 14 11:50 UTC] No.7448692[source]▶
I haven't visited Slashdot on a regular basis in the better part of ten years, but it tended to 15 years ago. Back then, it was the kind of place where you'd find people like John Carmack randomly chiming in on a thread.^1

And that's also entirely beside the point.

Does HN have the same level of comment quality as reddit? They both use simple up/down systems.

The point is that Slashdot's metamoderation system better leverages the community as a means of directing discussion. The initial upvote has a reason attached to it, and the meta level moderation allows the community to filter out people that are moderating inappropriately. What constitutes appropriate is going to depend on the standards in place for that community.

I wouldn't expect to see an option to moderate a comment +1, Funny on HN, for example. The system itself, however, is designed specifically to bring quality content to the top.

But I think you're making the mistake of conflating the design of a system that is meant to address one problem with the state of a site that declined for other reasons years ago.

[1] - http://floodyberry.com/carmack/slashdot.html#s19991123x08382...

replies(1): >>7449030 #
3. philh ◴[22 Mar 14 13:49 UTC] No.7449030[source]▶
Perhaps I misinterpreted you. When you said

> It baffles me that every site on the web keeps using simple up/down systems and then complains about how hard it is to generate quality discussion when this is hardly a new problem set.

It sounded like you were saying "if those sites just used a better moderation system, it would be easy to generate quality discussion". But the moderation system you propose does not generate quality discussion on slashdot.

If that's not what you're saying, then I'm not sure what particular benefit you think slashdot's sytem does have. What does it mean for a system to be good at directing discussion, but not at generating high-quality discussion? Why is it useful to be able to revoke mod privileges, if that doesn't generate high-quality discussion?

Or are you saying something like, it would be easy to generate quality discussion with slashdot's system given certain other factors (where simple +/- does not make this easy), but slashdot lacks those factors?

4. doorhammer ◴[22 Mar 14 16:08 UTC] No.7449575[source]▶
You bring up another interesting point for me. How do you even know when the problem is or isn't solved? There are two common themes I see in forums like this (and of course, many others of varying nuance). One group has been around for a long time and says it's just not like the old days, and a second group says that it is, and that the former are looking at the past through rose tinted glasses.

I can't objectively say that slashdot was better or worse here. Firstly, because I didn't go to slashdot nearly as often as I do here, and secondly, because I haven't done anything to thoroughly define and measure what I would consider a quality community.

If my problem is that the community is bad, I think I need a way to show to myself that it really is. I need a set of guidelines which I want my community to interact according to, or standards for the goodness I'm trying to push, and a way to see how much goodness there is, then a plan to drive that goodness.

And I'm not even saying that you need to be rigorous about it. Maybe that would help, but I think you can get some ideas with some general notions like tiny surveys, or something like that. I guess you could start getting more interesting and use sentiment analysis after auto-classifying posts, looking for meta content talking about the board.

As another aside, I think more specialized, smaller fragmented communities tend to be better for me, which is why I also enjoy subreddits like r/types and such, but again, I think that by no objective standard