It only ever happened when Draco himself said them, and that was rare enough. It only spurred his anger and jealousy. If this continued, Lucio would ask his father questions innocently, and then she would pay for it later.