multilinguality:multilingual language:zxx language:pes license:cc0-1.0 Glot Glot500 GlotLID multilingual LangID LID Language Identification region:us
    GlotLID Corpus \
---
multilinguality:
  - multilingual

extra_gated_heading: Access GlotLID Corpus on Hugging Face
extra_gated_description: >-
  This is a form to enable access to GlotLID Corpus on Hugging Face. Please email amir@cis.lmu.de with subject of "GlotLID Corpus", a form will be sent to you and after agreeing with our
  license terms and use policy your requests to access this repository will be accepted. Requests
  will be processed in 1-2 days.
extra_gated_prompt: "**Your Hugging Face account email address MUST match the email you provided us in form, or your request will not be approved.**"
extra_gated_button_content: Submit
extra_gated_fields:
  Name: text
  Email: text
  Affiliation: text
  Country: text
  Usecase: text
  I will only use this corpus myself for the use case that I have described above: checkbox


license: cc0-1.0
language:
- zxx
- pes

pretty_name: GlotLID Corpus
tags:
- Glot
- Glot500
- GlotLID
- multilingual
- LangID
- LID
- Language Identification
---

# GlotLID Corpus


## License
We do not own any of the text from which these data has been extracted.
We license the actual packaging, the metadata and the annotations of these data under the cc0-1.0 (waiving all of the rights under copyright law).

If you are a website/dataset owner and do not want your data to be included in this corpra, please send us an email at amir@cis.lmu.de .


## Ethical Considerations

**1. Biases:** The text corpus may reflect the perspectives, opinions, or demographics of its sources or creators. It is important for users to critically evaluate the text in context.

**2. Representativeness:** While we have aimed for diversity and inclusivity, the text corpus may not fully represent all native speakers. Users should be mindful of any potential underrepresentation.

**3. Ethics:** We acknowledge that the collection and use of text data can have ethical implications. We have strived to handle the data responsibly, but we encourage users to consider the broader ethical implications of their own research or applications.



## Citation
If you use any part of this code and collected data in your research, please cite it using the following BibTeX entry.

```
@inproceedings{
  kargaran2023glotlid,
  title={{GlotLID}: Language Identification for Low-Resource Languages},
  author={Kargaran, Amir Hossein and Imani, Ayyoob and Yvon, Fran{\c{c}}ois and Sch{\"u}tze, Hinrich},
  booktitle={The 2023 Conference on Empirical Methods in Natural Language Processing},
  year={2023},
  url={https://openreview.net/forum?id=dl4e3EBz5j}
}
```