Watermarks have been used for various purposes. Recently, researchers started to look into using them for deep neural networks. Some works try to hide attack triggers on their adversarial samples when attacking neural networks and others want to watermark neural networks to prove their ownership against plagiarism. Implanting a backdoor watermark module into a neural network is getting more attention from the community. In this paper, we present a general purpose encoder-decoder joint training method, inspired by generative adversarial networks (GANs). Unlike GANs, however, our encoder and decoder neural networks cooperate to find the best watermarking scheme given data samples. In other words, we do not design any new watermarking strategy but our proposed two neural networks will find the best suited method on their own. After being trained, the decoder can be implanted into other neural networks to attack or protect them (see Appendix for their use cases and real implementations). To this end, the decoder should be very tiny in order not to incur any overhead when attached to other neural networks but at the same time provide very high decoding success rates, which is very challenging. Our joint training method successfully solves the problem and in our experiments maintain almost 100\% encoding-decoding success rates for multiple datasets with very little modifications on data samples to hide watermarks. We also present several real-world use cases in Appendix. Security issues of deep learning have been very actively being studied. It had been already demonstrated that deep learning methods are vulnerable to some carefully devised adversarial attacks BID7 BID4 BID6 BID1 . At the same time, many researchers are also studying about how to make them more robust against such attacks. A couple of recent works, for example, proposed to use watermarks BID9 BID0 to protect neural networks. At the same time, other work wanted to use a similar watermark technique to attack neural networks BID9 .The method of adding watermarks to data samples can be used in various ways to protect deep learning models. First , the decoder can be implanted into a trained deep learning model and later one can prove the ownership, when other people copied the model, by showing that the copied model reacts to one's watermarked samples. Second , the implanted decoder may allow only legitimately watermarked samples and reject other non-watermarked samples. In this case, only people that have the encoder can access the deep learning model. However , there is one very strict requirement that the decoder should be tiny to minimize the incurred overheads by attaching it as part of the main deep learning model. Similar techniques can also be used to attack neural networks.In this paper, we do not propose any specific watermarking techniques. Instead , we want the encoder and decoder discuss and decide their watermarking method. Inspired from generative adversarial networks (GANs) BID3 , the encoder and decoder work for the same goal and are jointly trained. They do not perform the adversarial game of GANs. Their relationship is rather cooperative than adversarial in our method. The decoder is a tiny neural network to decode watermarks and the encoder is a high-capacity neural network that can watermark samples in such a way that the tiny neural network can successfully decode. Therefore, those two neural networks should cooperate to find such a watermarking scheme -in GANs, one neural network (generator) tries to fool the other neural network (discriminator). Because the decoder has a limited capacity due to its tiny neural network size, the encoder should not decide the watermarking scheme alone. The encoder should receive feedback from the decoder to revise its watermarking scheme. After training them , one should keep the encoder in a secure place but can deploy the decoder to as many places as one wants. We also show that our method can be used for both defences and attacks (refer to Appendix for some of these examples we implemented using our proposed method).We adopt residual blocks BID5 to design the encoder. Each residual block of the encoder is supposed to learn f (x)+x where x is an input to the block. One can consider f (x) as a watermark signal discovered by the joint training of the encoder and the decoder. The signal produced by f (x) should be strong enough to be detected by the decoder but weak enough not to be detected by human eyes. We design our training loss definition to achieve this goal. The encoder should modify original samples to implant watermarks. As more modifications are allowed, stronger watermarks will be implanted but they can be readily detected by human eyes. Our loss definition has a parameter that can be set by user to limit the modifications by the encoder. Our experiments show that we can find a well-balanced watermarking scheme that be detected only by the decoder.We tested many different datasets: face recognition(VGG-Face Data-set), speech recognition BID11 , images with general objects BID7 , and flowers (Flowers Data-set). Two of them are reported in the main paper with the comparison with other watermarking methods and others are introduced in Appendix. During experiments, our methods marked 100% decoding success rates for all datasets (in at least one hyper-parameter configuration). This well outperforms other baseline methods.In addition, we also found that different watermarking schemes are trained for different datasets. For instance, the encoder modified the tone of colors for the face recognition images. For the general object images, however , the encoder explicitly marks some dots rather than modifying their color tones (see FIG1 and Figure 4 ). This proves our goal that two neural networks cooperate to find the best suited watermarking method for each dataset. We present a joint training method of the watermark encoder and decoder. Our decoder is a very lowcapacity neural network and the encoder is a very high-capacity neural network. These two skinny and fatty neural networks collaborate to find the best watermarking scheme given data samples. In particular, we use residual blocks to build the encoder because the definition of the residual block is very appropriate for the task of watermarking samples. We demonstrated that two different types of watermarks (one to change the color tone and the other to add dots) are found by them without human interventions.For our experiments with various datasets, our method marked 100% decoding success rates, which means our tiny decoder is able to distinguish watermarked and non-watermarked samples perfectly.We also listed three use cases in Appendix about how to utilize our proposed encoder and decoder for real-world attacks and defenses. Our future research will be to implement those use cases. Figure 4: Examples of watermarking ImageNet images. Some dots are marked explicitly to hide watermarks when γ >= 0.05. Recall that watermarks are hidden in the tone of colors for FR images. This is a very interesting point because our proposed method can discover two very different watermarking schemes for them. This is because adding dots does not make the regularization term greatly exceed the margin γ. When γ = 0.01, a similar watermarking scheme to the FR exmaples will be used. This proves that our method is able to fine the best suited watermarking scheme given data samples. The decoder has 3 convolution layers in these examples. Note that there are more modifications in general as γ increases. For all cases, the trained decoder can successfully decode their watermarks. Figure 5 : The decoding success rate in the ImageNet dataset. We report the decoding success rate for non-watermarked/watermarked cases with our method after varying the convolution numbers in the decoder (i.e. decoder size) and γ.Our method Decoder size = 1 Decoder size = 3 γ = 0.01 γ = 0.05 γ = 0.1 γ = 0.01 γ = 0.05 γ = 0.1 81.2%/100.0% 89.2%/100.0% 92.0%/100.0% 99.0%/100.0% 98.0%/99.4% 99.5%/100.0% A ADDITIONAL EXPERIMENT RESULTS
We propose a novel watermark encoder-decoder neural networks. They perform a cooperative game to define their own watermarking scheme. People do not need to design watermarking methods any more.
{"src": "scitldr-AIC_train", "paper_id": "ryfDoiR5Ym"}