Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Spaces:

huggingface
/

text-data-filtering

Running

App Files Files Community

text-data-filtering

6 contributors

History: 57 commits

HugoLaurencon's picture

new kenlm models

b37a555 almost 3 years ago

.gitattributes

1.39 kB

chinese visu almost 3 years ago
.gitignore

25 Bytes

new tool to analyse our own doc almost 3 years ago
LICENSE

11.4 kB

Create LICENSE almost 3 years ago
README.md

909 Bytes

initial commit almost 3 years ago
app.py

42.4 kB

add register information almost 3 years ago
en.arpa.bin

4.44 GB
LFS

new kenlm models almost 3 years ago
en.sp.model

931 kB
LFS

new kenlm models almost 3 years ago
en_examples_with_stats.json

276 MB
LFS

new kenlm models almost 3 years ago
explanation_filtering_pipeline.pdf

217 kB

new filter on word repetition ratio almost 3 years ago
filtering.py

30.6 kB

rename badwords to flagged words + new flagged words list of 68 words almost 3 years ago
flagged_words.py

51.8 kB

rename badwords to flagged words + new flagged words list of 68 words almost 3 years ago
languages_id.py

5.61 kB

rename badwords to flagged words + new flagged words list of 68 words almost 3 years ago
lid.176.bin

131 MB
LFS

test almost 3 years ago
normalization.py

941 Bytes

test almost 3 years ago
parameters_filtering.py

31.7 kB

rename badwords to flagged words + new flagged words list of 68 words almost 3 years ago
requirements.txt

76 Bytes

fix requirements almost 3 years ago
stopwords.py

99.2 kB

test almost 3 years ago
zh.arpa.bin

3.64 GB
LFS

new kenlm models almost 3 years ago
zh.sp.model

876 kB
LFS

new kenlm models almost 3 years ago
zh_examples_with_stats.json

74.9 MB
LFS

new kenlm models almost 3 years ago