Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Punctuator for Uncased English
|
2 |
+
|
3 |
+
The model is fine-tuned based on `DistilBertForTokenClassification` for adding punctuations to plain text (uncased English)
|
4 |
+
|
5 |
+
## Usage
|
6 |
+
|
7 |
+
```python
|
8 |
+
from transformers import DistilBertForTokenClassification, DistilBertTokenizerFast
|
9 |
+
|
10 |
+
model = DistilBertForTokenClassification.from_pretrained("Qishuai/distilbert_punctuator_en")
|
11 |
+
tokenizer = DistilBertTokenizerFast.from_pretrained("Qishuai/distilbert_punctuator_en")
|
12 |
+
```
|
13 |
+
|
14 |
+
## Model Overview
|
15 |
+
|
16 |
+
### Training data
|
17 |
+
Combination of following three dataset:
|
18 |
+
|
19 |
+
- BBC news: From BBC news website corresponding to stories in five topical areas from 2004-2005. [Reference](https://www.kaggle.com/hgultekin/bbcnewsarchive)
|
20 |
+
- News articles: 20000 samples of short news articles scraped from Hindu, Indian times and Guardian between Feb 2017 and Aug 2017 [Reference](https://www.kaggle.com/sunnysai12345/news-summary?select=news_summary_more.csv)
|
21 |
+
- Ted talks: transcripts of over 4,000 TED talks between 2004 and 2019 [Reference](https://www.kaggle.com/miguelcorraljr/ted-ultimate-dataset)
|
22 |
+
|
23 |
+
### Model Performance
|
24 |
+
- Validation with 500 samples of dataset scraped from https://www.thenews.com.pk website. [Reference](https://www.kaggle.com/asad1m9a9h6mood/news-articles)
|
25 |
+
- Metrics Report:
|
26 |
+
|
27 |
+
| | precision | recall | f1-score | support |
|
28 |
+
|:--------------:|:---------:|:------:|:--------:|:-------:|
|
29 |
+
| COMMA | 0.66 | 0.55 | 0.60 | 7064 |
|
30 |
+
| EXLAMATIONMARK | 1.00 | 0.00 | 0.00 | 5 |
|
31 |
+
| PERIOD | 0.73 | 0.63 | 0.68 | 6573 |
|
32 |
+
| QUESTIONMARK | 0.54 | 0.41 | 0.47 | 17 |
|
33 |
+
| micro avg | 0.69 | 0.59 | 0.64 | 13659 |
|
34 |
+
| macro avg | 0.73 | 0.40 | 0.44 | 13659 |
|
35 |
+
| weighted avg | 0.69 | 0.59 | 0.64 | 13659 |
|
36 |
+
|
37 |
+
|
38 |
+
- Validation with 86 news ted talks of 2020 which are not included in training dataset [Reference](https://www.kaggle.com/thegupta/ted-talk)
|
39 |
+
- Metrics Report:
|
40 |
+
| | precision | recall | f1-score | support |
|
41 |
+
|:--------------:|:---------:|:------:|:--------:|:-------:|
|
42 |
+
| COMMA | 0.71 | 0.56 | 0.63 | 10712 |
|
43 |
+
| EXLAMATIONMARK | 0.45 | 0.07 | 0.12 | 75 |
|
44 |
+
| PERIOD | 0.75 | 0.65 | 0.70 | 7921 |
|
45 |
+
| QUESTIONMARK | 0.73 | 0.67 | 0.70 | 827 |
|
46 |
+
| micro avg | 0.73 | 0.60 | 0.66 | 19535 |
|
47 |
+
| macro avg | 0.66 | 0.49 | 0.53 | 19535 |
|
48 |
+
| weighted avg | 0.73 | 0.60 | 0.66 | 19535 |
|
49 |
+
|
50 |
+
|