Recent advances in Generative Adversarial Networks facilitated by improvements to the framework and successful application to various problems has resulted in extensions to multiple domains. IRGAN attempts to leverage the framework for Information-Retrieval (IR), a task that can be described as modeling the correct conditional probability distribution p(d|q) over the documents (d), given the query (q). The work that proposes IRGAN claims that optimizing their minimax loss function will result in a generator which can learn the distribution, but their setup and baseline term steer the model away from an exact adversarial formulation, and this work attempts to point out certain inaccuracies in their formulation. Analyzing their loss curves gives insight into possible mistakes in the loss functions and better performance can be obtained by using the co-training like setup we propose, where two models are trained in a co-operative rather than an adversarial fashion. Information-Retrieval (IR) involves providing a list of ranked documents {d 1 , d 2 , . . . , d k } in answer to a query q. This general formulation can be extended to various tasks like web-search, where the documents are web pages and information needs are queries, content-recommendation, where the documents are items/content to suggest and queries are users, and Question-Answering, where the documents are answers and queries are questions. The retrieved list can also be viewed as a probability distribution over candidates, one example being DISPLAYFORM0 where l is a hyperparameter. Even if the probability distribution is not explicit, it is desirable to retrieve a higher ranked document more often than a lower ranked document.GANs were proposed as alternatives to generative models and have been shown to be capable of modeling the true data well. High dimensional settings like images and word sequences have seen some success. Given that the generator in GANs tries to model the training data's distribution, adversarial setups seem like a natural fit for IR. The learned distribution can then be used to retrieve relevant documents for incoming queries. IRGAN is a framework proposed by , with the hope of giving Information-Retrieval, access to the large literature of GANs.IRGAN consists of a discriminator and a generator. Like in a typical setup, the discriminator learns to distinguish between documents produces by the real probability distribution or the real ranking and the generator's probability distribution. It increases the likelihood of the former and decreases it for the latter. The generator tries to bring its probability distribution closer to the real one so that it increases the likelihood of confusing the discriminator into believing that it is the true distribution. Ideally, equilibrium is achieved when the generator manages to rank the documents according to the true distribution.However, the formulation and implementation of the loss function in the work seems to have a few issues. Specifically, the use of the baseline term recommended in the work results in pitting the loss functions of the discriminator and the generator directly against each other and this leads to issues that are conspicuous in the loss curves. The training starts off with a pre-trained discriminator and generator, and the performance of the generator decreases as the training proceeds, while you would actually expect the opposite. When pre-training is not used, the generator does not learn at all. This forces IRGAN to choose the generator or discriminator based on whichever has better performance, while it expected that the generator is chosen at equilibrium.Given the traction this paper has received since its inception (53 citations as of 27 th September 2018), it is important to critically analyze the work and attribute the claimed performance improvements correctly. To this end, we propose two models which outperform IRGAN on two of the three tasks and give a comparable performance on the third. They also serve as an ablation study by experimentally showing that the generator might not be playing a vital role during train or test time.The following contributions are made in this work• We propose a model motivated by Co-training which outperforms IRGANs • We point out inaccuracies in the minimax loss function used in IRGANs • We substantiate the same by drawing conclusions from the loss curves 2 RELATED WORK 2.1 GENERATIVE ADVERSARIAL NETWORKS Generative Adversarial Networks (GANs) BID11 ) were proposed as an alternative to generative models BID23 ) which used Markov Chains or other approximations to compute intractable probability distributions. In essence, the generator tries to model the real data distribution and the discriminator learns to differentiate between real data points and generated data points. GANs are notoriously unstable to train and works like DCGANs BID21 ) and Wasserstein GAN BID1 ) have successfully attempted to alleviate a few issues. Nonetheless, GANs have been widely applied to various problems like image generation, text generation, cross-modal retrieval and more niche ones like Interactive Image Generation BID29 ), Text to Image ), Image to Image style transfer BID12 ) and robotics BID4 ).While GANs allow generation based on a random variable z, Conditional GANs BID17 ) partition the sample variable into two parts (z and y). y is used to denote which part of the probability distribution the generator has to generate from, and z plays the same role played in Vanilla GANs BID11 ). Conditional GANs dovetail with IR because y can be used to represent the query or its embedding, and in theory, the model should be able to generate the required document. DISPLAYFORM1 We feel that an eventual adversarial formulation for IR will be similar to this in flavor. The experiments performed show that IRGAN is by no means state-of-the-art on those datasets. Further, the performance does not justify the large training time of 4 hours per generator epoch and 1 hour of discriminator epoch as opposed to 2 hours per epoch of the co-training model (11 GB GPU and Question Answering task). The shaky mathematical formulation renders the generator useless after training, and any gains in performance can be attributed directly to the first term of J D , where the likelihood of the real data is increased. We showed that the discriminator and generator are optimizing directly opposite loss functions and this is the cause of deleterious training.The poor performance of IRGAN on Web-Search and Question Answering and only a satisfactory performance on Content-Recommendation (which has dense rewards) lead us to speculate that it does not work well in sparse reward scenarios. This is similar to a well-known problem called the Sparse Reward Reinforcement Learning. We think that a correct formulation along with established techniques from the former, like reward shaping BID18 ) may lead to better performance. Newer methods like Hindsight Experience Replay BID0 ) which allow models to learn both from mistakes and rewards may further ameliorate learning.We would also like to explore in the direction of learning correct adversarial frameworks for more complex tasks like Image Retrieval and Question Answering which will involve learning end-toend trainable models. With advances in modeling sequences, this could also involve generation of documents rather than sampling them.
Points out problems in loss function used in IRGAN, a recently proposed GAN framework for Information Retrieval. Further, a model motivated by co-training is proposed, which achieves better performance.
{"src": "scitldr-AIC_train", "paper_id": "Syez3j0cKX"}