←back to thread

‘Overworked, underpaid’ humans train Google’s AI

(www.theguardian.com)
283 points Brajeshwar | 4 comments | 13 Sep 25 11:30 UTC | HN request time: 0.001s | source
Show context
iandanforth ◴[13 Sep 25 12:41 UTC] No.45231600[source]▶
"Google said in a statement: “Quality raters are employed by our suppliers and are temporarily assigned to provide external feedback on our products. Their ratings are one of many aggregated data points that help us measure how well our systems are working, but do not directly impact our algorithms or models.” GlobalLogic declined to comment for this story." (emphasis mine)

How is this not a straight up lie? For this to be true they would have to throw away labeled training data.

replies(4): >>45231651 #>>45231697 #>>45231758 #>>45232359 #
1. yobbo ◴[13 Sep 25 14:30 UTC] No.45232359[source]▶
> For this to be true they would have to throw away labeled training data.

That's how validation works.

replies(1): >>45233162 #
2. jfengel ◴[13 Sep 25 16:03 UTC] No.45233162[source]▶
Is there a reason not to use validation data in your next round of training data? Or is it more efficient to reuse validation and instead get more training data?
replies(1): >>45233504 #
3. parineum ◴[13 Sep 25 16:53 UTC] No.45233504[source]▶
You'd have to recreate your validation if you trained your model on it every iteration and then they wouldn't be consistent enough to show any trends
replies(1): >>45240383 #
4. jfengel ◴[14 Sep 25 15:13 UTC] No.45240383{3}[source]▶
I'd have thought that if you kept the same validation you'd risk over fitting.

Clearly that does make it hard to measure. I'd think you'd want "equivalent" validation (like changing the SATs every year), though I imagine that's not really a meaningful concept.