Update README.md

README.md

@@ -1,38 +1,78 @@
 ---
+license: apache-2.0
 language:
 - ja
-license: apache-2.0
 tags:
 - automatic-speech-recognition
-- mozilla-foundation/common_voice_8_0
-- generated_from_trainer
+- robust-speech-event
+- common-voice
+- ja
 model-index:
-- name: wav2vec2-xls-r-1b-ja
-  results: []
+- name: wav2vec2-xls-r-1b
+  results:
+  - task: 
+      name: Speech Recognition
+      type: automatic-speech-recognition
+    dataset:
+      name: Common Voice 7.0
+      type: mozilla-foundation/common_voice_7_0
+      args: ja
+    metrics:
+       - name: Test WER (with LM)
+         type: wer
+         value: 11.77
+       - name: Test CER (with LM)
+         type: cer
+         value: 5.22
+  - task:
+      name: Speech Recognition
+      type: automatic-speech-recognition
+    dataset:
+      name: Common Voice 8.0
+      type: mozilla-foundation/common_voice_8_0
+      args: ja
+    metrics:
+       - name: Test WER (with LM)
+         type: wer
+         value: 12.23
+       - name: Test CER (with LM)
+         type: cer
+         value: 5.33
+  - task:
+      name: Speech Recognition
+      type: automatic-speech-recognition
+    dataset:
+      name: Robust Speech Event - Dev Data
+      type: speech-recognition-community-v2/dev_data
+      args: ja
+    metrics:
+       - name: Test WER (with LM)
+         type: wer
+         value: 29.35
+       - name: Test CER (with LM)
+         type: cer
+         value: 16.43
 ---
-
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
-# wav2vec2-xls-r-1b-ja
-
-This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - JA dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.3316
-- Wer: 0.2564
-- Cer: 0.1218
-
 ## Model description
 
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
+This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - JA 
+
+### Benchmark WER result:
+| | [COMMON VOICE 7.0](https://huggingface.co/datasets/mozilla-foundation/common_voice_7_0) | [COMMON VOICE 8.0](https://huggingface.co/datasets/mozilla-foundation/common_voice_8_0)
+|---|---|---|
+|without LM| 16.97 | 17.95 |
+|with 4-grams LM| 11.77 | 12.23|
+### Benchmark CER result:
+| | [COMMON VOICE 7.0](https://huggingface.co/datasets/mozilla-foundation/common_voice_7_0) | [COMMON VOICE 8.0](https://huggingface.co/datasets/mozilla-foundation/common_voice_8_0)
+|---|---|---|
+|without LM| 6.82 | 7.05 |
+|with 4-grams LM| 5.22 | 5.33 |
+## Evaluation
+Please use the eval.py file to run the evaluation:
+```python
+pip install mecab-python3 unidic-lite pykakasi
+python eval.py --model_id vutankiet2901/wav2vec2-xls-r-1b-ja --dataset mozilla-foundation/common_voice_7_0 --config ja --split test --log_outputs
+```
 
 ## Training procedure