←back to thread

Changes since congestion pricing started in New York

(www.nytimes.com)
437 points Vinnl | 1 comments | 13 May 25 10:43 UTC | HN request time: 0.25s | source
Show context
explodes ◴[14 May 25 13:18 UTC] No.43984193[source]▶
Wouldn't it be nice if policy changes were accompanied by an A/B testing plan to evaluate their impact? I have always thought so. I have also seen a major pitfall of A/B testing that real humans can hand-pick and slice data to make it sound as positive or negative as wanted. Nonetheless, the more data the better.
replies(9): >>43984911 #>>43984957 #>>43985195 #>>43985261 #>>43985635 #>>43986010 #>>43986433 #>>43990930 #>>43996358 #
Calwestjobs ◴[14 May 25 14:47 UTC] No.43985195[source]▶
test A - before

test B - after

what are you talking about ?

replies(3): >>43985239 #>>43985257 #>>43985425 #
1. Ntrails ◴[14 May 25 15:06 UTC] No.43985425[source]▶
"before" and "after" introduces a large axis of noise

The problem is that for A/B testing to really work you need independent groups outcomes. As soon as there is any bias in group selection or cross group effect it's very hard to unpick.