I kind of see myself as the balancer. I tend to actually read a downvoted comment to see if it was worthy (and vote it up) or see a highly upvoted comment and possibly vote it down if it's just getting upvoted because it puts forward a popular view (e.g. Microsoft is evil) rather than something of value.
If downvotes are used for disagreement, then I have to read them in order to know if they are relevant. If downvotes are saved only for irrelevant, one-liners, me-too's, etc., then I can safely ignore them.
