No, because that allows one person to double down on negativity - it can turn something grey
and detract from the comment, both of which then bias future readers to read it negatively.
I'd go the other way - disallow downvoting if you comment, and disallow comments if you downvoted. You get one choice of how to indicate that you don't feel the comment is correct.
Admittedly, teaching people not to downvote just for disagreement would be better. Personally, I downvote a ton, but never for disagreement - I downvote for exactly the reasons I see in the guidelines: non-substantive comments. Or, admittedly, I also downvote people who are just being a jerk.