Qishuai commited on
Commit
3b5050a
1 Parent(s): 067db2e

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -0
README.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Punctuator for Uncased English
2
+
3
+ The model is fine-tuned based on `DistilBertForTokenClassification` for adding punctuations to plain text (uncased English)
4
+
5
+ ## Usage
6
+
7
+ ```python
8
+ from transformers import DistilBertForTokenClassification, DistilBertTokenizerFast
9
+
10
+ model = DistilBertForTokenClassification.from_pretrained("Qishuai/distilbert_punctuator_en")
11
+ tokenizer = DistilBertTokenizerFast.from_pretrained("Qishuai/distilbert_punctuator_en")
12
+ ```
13
+
14
+ ## Model Overview
15
+
16
+ ### Training data
17
+ Combination of following three dataset:
18
+
19
+ - BBC news: From BBC news website corresponding to stories in five topical areas from 2004-2005. [Reference](https://www.kaggle.com/hgultekin/bbcnewsarchive)
20
+ - News articles: 20000 samples of short news articles scraped from Hindu, Indian times and Guardian between Feb 2017 and Aug 2017 [Reference](https://www.kaggle.com/sunnysai12345/news-summary?select=news_summary_more.csv)
21
+ - Ted talks: transcripts of over 4,000 TED talks between 2004 and 2019 [Reference](https://www.kaggle.com/miguelcorraljr/ted-ultimate-dataset)
22
+
23
+ ### Model Performance
24
+ - Validation with 500 samples of dataset scraped from https://www.thenews.com.pk website. [Reference](https://www.kaggle.com/asad1m9a9h6mood/news-articles)
25
+ - Metrics Report:
26
+
27
+ | | precision | recall | f1-score | support |
28
+ |:--------------:|:---------:|:------:|:--------:|:-------:|
29
+ | COMMA | 0.66 | 0.55 | 0.60 | 7064 |
30
+ | EXLAMATIONMARK | 1.00 | 0.00 | 0.00 | 5 |
31
+ | PERIOD | 0.73 | 0.63 | 0.68 | 6573 |
32
+ | QUESTIONMARK | 0.54 | 0.41 | 0.47 | 17 |
33
+ | micro avg | 0.69 | 0.59 | 0.64 | 13659 |
34
+ | macro avg | 0.73 | 0.40 | 0.44 | 13659 |
35
+ | weighted avg | 0.69 | 0.59 | 0.64 | 13659 |
36
+
37
+
38
+ - Validation with 86 news ted talks of 2020 which are not included in training dataset [Reference](https://www.kaggle.com/thegupta/ted-talk)
39
+ - Metrics Report:
40
+ | | precision | recall | f1-score | support |
41
+ |:--------------:|:---------:|:------:|:--------:|:-------:|
42
+ | COMMA | 0.71 | 0.56 | 0.63 | 10712 |
43
+ | EXLAMATIONMARK | 0.45 | 0.07 | 0.12 | 75 |
44
+ | PERIOD | 0.75 | 0.65 | 0.70 | 7921 |
45
+ | QUESTIONMARK | 0.73 | 0.67 | 0.70 | 827 |
46
+ | micro avg | 0.73 | 0.60 | 0.66 | 19535 |
47
+ | macro avg | 0.66 | 0.49 | 0.53 | 19535 |
48
+ | weighted avg | 0.73 | 0.60 | 0.66 | 19535 |
49
+
50
+