←back to thread

Auto-grading decade-old Hacker News discussions with hindsight

(karpathy.bearblog.dev)
626 points __rito__ | 1 comments | 10 Dec 25 17:23 UTC | HN request time: 0.195s | source

Related from yesterday: Show HN: Gemini Pro 3 imagines the HN front page 10 years from now - https://news.ycombinator.com/item?id=46205632
Show context
popinman322 ◴[11 Dec 25 04:43 UTC] No.46227755[source]▶
It doesn't look like the code anonymizes usernames when sending the thread for grading. This likely induces bias in the grades based on past/current prevailing opinions of certain users. It would be interesting to see the whole thing done again but this time randomly re-assigning usernames, to assess bias, and also with procedurally generated pseudonyms, to see whether the bias can be removed that way.

I'd expect de-biasing would deflate grades for well known users.

It might also be interesting to use a search-grounded model that provides citations for its grading claims. Gemini models have access to this via their API, for example.

replies(2): >>46228238 #>>46231628 #
1. ProllyInfamous ◴[11 Dec 25 14:15 UTC] No.46231628[source]▶
What a human-like critizicism of human-like behavior.

I [as a human] also do the same thing when observing others in IRL and forum interactions. Reputation matters™

----

A further question is whether a bespoke username could influence the bias of a particular comment (e.g. A username of something like HatesPython might influence the interpretation of that commenter's particular perception of the Python coding language, which might actually be expressing positivity — the username's irony lost to the AI?).