
# Hotwords and ngram files used in wenet.
```
.
├── models
│   └── hotwords.yaml
```

# Evaluation
Base Acoustic model: [20210601_u2++_conformer_exp](https://wenet-1256283475.cos.ap-shanghai.myqcloud.com/models/aishell/20210601_u2%2B%2B_conformer_exp.tar.gz)
## Tested ENV
* CPU：40 Core, Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz
* GPU：NVIDIA GeForce RTX 2080 Ti

[**AISHELL-1 Test dataset**](https://www.openslr.org/33/)
| model (FP16)                 | RTF     | CER    |
|------------------------------|---------|--------|
| offline model w/o hotwords   | 0.00437 | 4.6805 |
| offline model w/  hotwords   | 0.00428 | 4.5841 |
| streaming model w/o hotwords | 0.01231 | 5.2777 |
| streaming model w/  hotwords | 0.01195 | 5.1850 |

[**AISHELL-1 hostwords testsets**](https://www.modelscope.cn/datasets/speech_asr/speech_asr_aishell1_hotwords_testsets/summary)
| model (FP16)               | Latency (s) | CER   | Recall | Precision | F1-score |
|----------------------------|-------------|-------|--------|-----------|----------|
| offline model w/o hotwords | 5.8673      | 13.85 | 0.27   | 0.99      | 0.43     |
| offline model w/  hotwords | 5.6601      | 11.96 | 0.47   | 0.97      | 0.63     |

# Decoding result
| Label                | hotwords  | pred w/o hotwords            | pred w/ hotwords             |
|----------------------|-----------|------------------------------|------------------------------|
| 以及拥有陈露的女单项目          | 陈露        | 以及拥有**陈鹭**的女单项目              | 以及拥有**陈露**的女单项目              |
| 庞清和佟健终于可以放心地考虑退役的事情了 | 庞清<br/>佟健 | **庞青**和**董建**终于可以放心地考虑退役的事情了 | **庞清**和**佟健**终于可以放心地考虑退役的事情了 |
| 赵继宏老板电器做厨电已经三十多年了    | 赵继宏       | **赵继红**老板电器做厨店已经三十多年了        | **赵继宏**老板电器做厨电已经三十多年了        |

**Result Files**
```
.
└── results
    ├── aishell1_testsets
    │   ├── offline_with_hotwords_orderAuto.log
    │   ├── offline_without_hotwords.log
    │   ├── stream_with_hotwords_orderAuto.log
    │   └── stream_without_hotwords.log
    └── speech_asr_aishell1_hotwords_testsets
        ├── aishell1_text_hotwords
        ├── with_hotwords_ali.log
        └── without_hotwords_ali.log
```
# NER evaluation script
```
cd ner

python compute_hotwords_f1.py \
    --label="data/aishell1_text_hotwords" \
    --preds="data/with_hotwords_ali.log;data/without_hotwords_ali.log" \
    --hotword="data/hotwords.yaml"
```


{"license": "apache-2.0"}
license:apache-2.0 region:us
TAGS
#license-apache-2.0 #region-us 

Hotwords and ngram files used in wenet.
=======================================


Evaluation
==========


Base Acoustic model: 20210601\_u2++\_conformer\_exp


Tested ENV
----------


* CPU：40 Core, Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz
* GPU：NVIDIA GeForce RTX 2080 Ti


AISHELL-1 Test dataset


model (FP16): offline model w/o hotwords, RTF: 0.00437, CER: 4.6805
model (FP16): offline model w/ hotwords, RTF: 0.00428, CER: 4.5841
model (FP16): streaming model w/o hotwords, RTF: 0.01231, CER: 5.2777
model (FP16): streaming model w/ hotwords, RTF: 0.01195, CER: 5.1850


AISHELL-1 hostwords testsets



Decoding result
===============



Result Files


NER evaluation script
=====================



TAGS
#license-apache-2.0 #region-us 
