Once we recognized the distinction between voting and flagging it seemed more natural to have this new feature be on the flagging side, hence vouches. The jury's still out on whether it's a good idea or not, though; if it turns out bad, we might consider reverting to the voting idea.
Regardless of the constant arguments around it, I use upvote/downvote on opinions I agree/disagree with. I'd use flag/vouch for things that I don't/do think should be on HN at all.
e.g., on a Pokemon forum I can strongly disagree with someone who says squirtel is the better than pikachu because water-type has more advantages that electric-type. I might downvote it, maybe. But if someone posts "Pokemon are stupid, you're all wasting your lives you big losers." ...I'd flag that. Conversely, if I posted how electric-type pokemon are the best and I notice a dead-reply to me explaining very clearly a position supporting water-type pokemon as best, I'd vouch for it. While I might not agree with the argument, I can at least vouch that it shouldn't be dead.