Add evaluation results on the squad_v2 config of squad_v2 (#1)

- Add evaluation results on the squad_v2 config of squad_v2 (afaec4df4fb9b4cef1694dc2961e45dccb216da2)


Co-authored-by: Evaluation Bot <[email protected]>

README.md

@@ -1,9 +1,31 @@
 ---
 widget:
-- context: While deep and large pre-trained models are the state-of-the-art for various natural language processing tasks, their huge size poses significant challenges for practical uses in resource constrained settings. Recent works in knowledge distillation propose task-agnostic as well as task-specific methods to compress these models, with task-specific ones often yielding higher compression rate. In this work, we develop a new task-agnostic distillation framework XtremeDistilTransformers that leverages the advantage of task-specific methods for learning a small universal model that can be applied to arbitrary tasks and languages. To this end, we study the transferability of several source tasks, augmentation resources and model architecture for distillation. We evaluate our model performance on multiple tasks, including the General Language Understanding Evaluation (GLUE) benchmark, SQuAD question answering dataset and a massive multi-lingual NER dataset with 41 languages. 
+- context: While deep and large pre-trained models are the state-of-the-art for various
+    natural language processing tasks, their huge size poses significant challenges
+    for practical uses in resource constrained settings. Recent works in knowledge
+    distillation propose task-agnostic as well as task-specific methods to compress
+    these models, with task-specific ones often yielding higher compression rate.
+    In this work, we develop a new task-agnostic distillation framework XtremeDistilTransformers
+    that leverages the advantage of task-specific methods for learning a small universal
+    model that can be applied to arbitrary tasks and languages. To this end, we study
+    the transferability of several source tasks, augmentation resources and model
+    architecture for distillation. We evaluate our model performance on multiple tasks,
+    including the General Language Understanding Evaluation (GLUE) benchmark, SQuAD
+    question answering dataset and a massive multi-lingual NER dataset with 41 languages.
   example_title: xtremedistil q1
   text: What is XtremeDistil?
-- context: While deep and large pre-trained models are the state-of-the-art for various natural language processing tasks, their huge size poses significant challenges for practical uses in resource constrained settings. Recent works in knowledge distillation propose task-agnostic as well as task-specific methods to compress these models, with task-specific ones often yielding higher compression rate. In this work, we develop a new task-agnostic distillation framework XtremeDistilTransformers that leverages the advantage of task-specific methods for learning a small universal model that can be applied to arbitrary tasks and languages. To this end, we study the transferability of several source tasks, augmentation resources and model architecture for distillation. We evaluate our model performance on multiple tasks, including the General Language Understanding Evaluation (GLUE) benchmark, SQuAD question answering dataset and a massive multi-lingual NER dataset with 41 languages.
+- context: While deep and large pre-trained models are the state-of-the-art for various
+    natural language processing tasks, their huge size poses significant challenges
+    for practical uses in resource constrained settings. Recent works in knowledge
+    distillation propose task-agnostic as well as task-specific methods to compress
+    these models, with task-specific ones often yielding higher compression rate.
+    In this work, we develop a new task-agnostic distillation framework XtremeDistilTransformers
+    that leverages the advantage of task-specific methods for learning a small universal
+    model that can be applied to arbitrary tasks and languages. To this end, we study
+    the transferability of several source tasks, augmentation resources and model
+    architecture for distillation. We evaluate our model performance on multiple tasks,
+    including the General Language Understanding Evaluation (GLUE) benchmark, SQuAD
+    question answering dataset and a massive multi-lingual NER dataset with 41 languages.
   example_title: xtremedistil q2
   text: On what is the model validated?
 datasets:
@@ -13,6 +35,26 @@ metrics:
 - exact
 tags:
 - question-answering
+model-index:
+- name: nbroad/xdistil-l12-h384-squad2
+  results:
+  - task:
+      type: question-answering
+      name: Question Answering
+    dataset:
+      name: squad_v2
+      type: squad_v2
+      config: squad_v2
+      split: validation
+    metrics:
+    - name: Exact Match
+      type: exact_match
+      value: 75.4591
+      verified: true
+    - name: F1
+      type: f1
+      value: 79.3321
+      verified: true
 ---
 
 xtremedistil-l12-h384 trained on SQuAD 2.0