DeepPavlov
/

distilrubert-tiny-cased-conversational

Inference Endpoints

Model card Files Files and versions Community

DeepPavlovAI commited on Jan 20, 2022

Commit

2c754a3

•

1 Parent(s): c5ae87f

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -2,8 +2,8 @@
 language:
 - ru
 ---
-# distilrubert-tiny-cased-conversational
-Conversational DistilRuBERT-tiny \(Russian, cased, 2‑layer, 768‑hidden, 12‑heads, 107M parameters\) was trained on OpenSubtitles\[1\], [Dirty](https://d3.ru/), [Pikabu](https://pikabu.ru/), and a Social Media segment of Taiga corpus\[2\] (as [Conversational RuBERT](https://huggingface.co/DeepPavlov/rubert-base-cased-conversational)). It can be considered as tiny copy of [Conversational DistilRuBERT-base](https://huggingface.co/DeepPavlov/distilrubert-base-cased-conversational).
 Our DistilRuBERT-tiny was highly inspired by \[3\], \[4\]. Namely, we used
 * KL loss (between teacher and student output logits)

 language:
 - ru
 ---
+# distilrubert-small-cased-conversational
+Conversational DistilRuBERT-small \(Russian, cased, 2‑layer, 768‑hidden, 12‑heads, 107M parameters\) was trained on OpenSubtitles\[1\], [Dirty](https://d3.ru/), [Pikabu](https://pikabu.ru/), and a Social Media segment of Taiga corpus\[2\] (as [Conversational RuBERT](https://huggingface.co/DeepPavlov/rubert-base-cased-conversational)). It can be considered as small copy of [Conversational DistilRuBERT-base](https://huggingface.co/DeepPavlov/distilrubert-base-cased-conversational).
 Our DistilRuBERT-tiny was highly inspired by \[3\], \[4\]. Namely, we used
 * KL loss (between teacher and student output logits)