<commit_before>---
layout: base
title:  'Introduction'
permalink: sl/overview/introduction.html
---

# Introduction

The Slovenian UD Treebank is based on the [ssj500k Dependency Treebank](http://eng.slovenscina.eu/tehnologije/ucni-korpus) (Krek and Erjavec 2014), a representative balanced collection of sampled texts from the reference FidaPLUS corpus of written Slovenian (Arhar and Gorjanc 2007) that have been manually segmented, tokenized, lemmatized and morphosyntactically tagged in accordance with the [JOS annotation scheme](http://nl.ijs.si/jos/index-en.html) (Erjavec et al. 2010). One half of the ssj500k corpus also includes manually annotated dependency relations and this is what has been used as the basis of the Slovenian UD Treebank.

The Slovenian UD Treebank thus includes 11.411 sentences with 235.864 tokens taken from a variety of text genres, such as fiction, non-fiction and periodicals, dating from 1990 to 2000. 

The current version of the Slovene UD Treebank (not yet officially released) includes automatic conversions of POS categories, features and values from JOS to UD, which have also been described in the corresponding [Slovenian UD Guidelines](http://universaldependencies.github.io/docs/#language-sl). Further revisions of this work and syntactic mapping remains a work in progress.

## Acknowledgments

We wish to thank all of the contributors to the original ssj500k training corpus: <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>.

## References

* <NAME> and <NAME>. 2014. Training corpus ssj500kv1.2. Jožef Stefan Institute, Slovenia. [http://hdl.handle.net/11356/4](http://hdl.handle.net/11356/4)
* <NAME> and <NAME>. 2007. Korpus FidaPLUS: nova generacija slovenskega referenčnega korpusa (The FidaPLUS corpus: a new generation of the Slovene reference corpus). Jezik in slovstvo, 52(2). 
* <NAME>, <NAME>, <NAME> and <NAME>. 2010. The JOS Linguistically Tagged Corpus of Slovene. In: Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10), Malta.

<commit_msg>Update introduction.md<commit_after>---
layout: base
title:  'Introduction'
permalink: sl/overview/introduction.html
---

# Introduction

The Slovenian UD Treebank is based on the [ssj500k Dependency Treebank](http://eng.slovenscina.eu/tehnologije/ucni-korpus) (Krek and Erjavec 2014), a representative balanced collection of sampled texts from the reference FidaPLUS corpus of written Slovenian (Arhar and Gorjanc 2007) that have been manually segmented, tokenized, lemmatized and morphosyntactically tagged in accordance with the [JOS annotation scheme](http://nl.ijs.si/jos/index-en.html) (Erjavec et al. 2010). One half of the ssj500k corpus also includes manually annotated dependency relations and this is what has been used as the basis of the Slovenian UD Treebank.

The Slovenian UD Treebank thus includes 11.411 sentences with 235.864 tokens taken from a variety of text genres, such as fiction, non-fiction and periodicals, dating from 1990 to 2000. 

The current version of the Slovene UD Treebank (not yet officially released) includes automatic conversions of POS categories, features and values from JOS to UD, which have also been described in the corresponding [Slovenian UD Guidelines](http://universaldependencies.github.io/docs/#language-sl). Further revisions of this work and syntactic mapping remains a work in progress.

## Acknowledgments

We wish to thank all of the contributors to the original ssj500k training corpus: <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>.

## References

* <NAME> and <NAME>. 2014. Training corpus ssj500kv1.2. Jožef Stefan Institute, Slovenia. 
* <NAME> and <NAME>. 2007. Korpus FidaPLUS: nova generacija slovenskega referenčnega korpusa (The FidaPLUS corpus: a new generation of the Slovene reference corpus). Jezik in slovstvo, 52(2). 
* <NAME>, <NAME>, <NAME> and <NAME>. 2010. The JOS Linguistically Tagged Corpus of Slovene. In: Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10), Malta.


{'fraction_non_alphanumeric': 0.10089224433768017, 'fraction_numerical': 0.03111416151910318, 'mean_word_length': 5.149085794655415, 'pattern_counts': {'":': 0, '<': 71, '<?xml version=': 0, '>': 71, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7192739', 'n_tokens_mistral': 1430, 'n_tokens_neox': 1244, 'n_words': 550}