That's what we expect, actually. If you don't like the comment, rate it as poor quality. For the civility vote, only rate it as "not civil" if it's a personal attack, harassment, or abuse. 

We didn't want people combining the two things, so we split it up into two questions - you can dislike a comment that doesn't break the rules.