As a consequence, CMC has seen a significant surge in research in recent years. In less than 2 years since the introduction of these datasets, there have been 40 submissionsRecorded as of July 1, 2020. Please note that many of these submissions are either ensemble versions of single models, or hyper-parameter variants of their pre-published models or are simply unpublished. Therefore, unique published models' count is 13 in CoQA and 7 in QuAC. to CoQA leaderboardhttps://stanfordnlp.github.io/coqa/ and 22 submissionscurrent to QuAC leaderboardhttp://quac.ai/. Many of these models are unpublished, indicating active ongoing research on these datasets. Besides, the current state-of-the-art in QuAC lags behind human performance F1 benchmark by a margin of 6.7current, suggesting significant scope for improvement. Almost simultaneously, there have been breakthroughs in NLP [1]}, [2]}, [3]} which the researchers have tried to leverage in their upcoming models [4]}, [5]}, [6]}.
Since many models are being published concurrently, there have been inconsistencies and/or overlap in their methodologies and research directions. This makes it difficult to compare different approaches against each other and weigh their pros and cons.
This prevailing scenario has blurred the bigger picture and made it difficult for researchers to attend to novel research in this field.
Moreover, there is no singular summarized view on CMC models, except the individual literature studies of these publications which can be highly localized and inconsistent with the global view. Thus, the current scenario motivates the need for organizing the scattered knowledge across these publications into a consolidated overview, so that future research in this field can be streamlined.
