Add model card (#1)

- Add model card (ce151e2111d25abf840d9e3c16570f53c50613ff)
- Update README.md (8170ea6e8b18f660ee3ea915006619e8ab838f4f)
- Update README.md (b8f0f9ed6e98177246f7c5bb014af410a767a53d)


Co-authored-by: Marissa Gerchick <[email protected]>

README.md

@@ -0,0 +1,215 @@
+---
+language:
+- multilingual
+- en
+- es
+- fr
+- de
+- zh
+- ru 
+- pt
+- it
+- ar
+- ja 
+- id 
+- tr 
+- nl
+- pl
+- fa
+- vi
+- sv
+- ko 
+- he
+- ro
+- no
+- hi
+- uk
+- cs
+- fi
+- hu
+- th
+- da
+- ca
+- el
+- bg
+- sr
+- ms
+- bn
+- hr
+- sl
+- az
+- sk
+- eo
+- ta
+- sh
+- lt
+- et
+- ml
+- la
+- bs
+- sq
+- arz
+- af
+- ka
+- mr
+- eu
+- tl
+- ang
+- gl
+- nn
+- ur
+- kk
+- be
+- hy
+- te
+- lv
+- mk
+- als
+- is
+- wuu
+- my
+- sco
+- mn
+- ceb
+- ast
+- cy
+- kn
+- br
+- an
+- gu
+- bar
+- uz
+- lb
+- ne
+- si
+- war
+- jv
+- ga
+- oc
+- ku
+- sw
+- nds
+- ckb
+- ia
+- yi
+- fy
+- scn
+- gan
+- tt
+- am
+license: cc-by-nc-4.0
+---
+
+# xlm-mlm-100-1280
+
+#  Table of Contents
+
+1. [Model Details](#model-details)
+2. [Uses](#uses)
+3. [Bias, Risks, and Limitations](#bias-risks-and-limitations)
+4. [Training](#training)
+5. [Evaluation](#evaluation)
+6. [Environmental Impact](#environmental-impact)
+7. [Technical Specifications](#technical-specifications)
+8. [Citation](#citation)
+9. [Model Card Authors](#model-card-authors)
+10. [How To Get Started With the Model](#how-to-get-started-with-the-model)
+
+
+# Model Details
+
+xlm-mlm-100-1280 is the XLM model, which was proposed in [Cross-lingual Language Model Pretraining](https://arxiv.org/abs/1901.07291) by Guillaume Lample and Alexis Conneau, trained on Wikipedia text in 100 languages. The model is a transformer pretrained using a masked language modeling (MLM) objective. 
+
+## Model Description
+
+- **Developed by:** See [associated paper](https://arxiv.org/abs/1901.07291) and [GitHub Repo](https://github.com/facebookresearch/XLM)
+- **Model type:** Language model
+- **Language(s) (NLP):** 100 languages, see [GitHub Repo](https://github.com/facebookresearch/XLM#the-17-and-100-languages) for full list.
+- **License:** CC-BY-NC-4.0
+- **Related Models:** [xlm-mlm-17-1280](https://huggingface.co/xlm-mlm-17-1280)
+- **Resources for more information:** 
+  - [Associated paper](https://arxiv.org/abs/1901.07291)
+  - [GitHub Repo](https://github.com/facebookresearch/XLM#the-17-and-100-languages)
+  - [Hugging Face Multilingual Models for Inference docs](https://huggingface.co/docs/transformers/v4.20.1/en/multilingual#xlm-with-language-embeddings)
+
+# Uses
+
+## Direct Use
+
+The model is a language model. The model can be used for masked language modeling. 
+
+## Downstream Use
+
+To learn more about this task and potential downstream uses, see the Hugging Face [fill mask docs](https://huggingface.co/tasks/fill-mask) and the [Hugging Face Multilingual Models for Inference](https://huggingface.co/docs/transformers/v4.20.1/en/multilingual#xlm-with-language-embeddings) docs. Also see the [associated paper](https://arxiv.org/abs/1901.07291).
+
+## Out-of-Scope Use
+
+The model should not be used to intentionally create hostile or alienating environments for people. 
+
+# Bias, Risks, and Limitations
+
+Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)).
+
+## Recommendations
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
+
+# Training
+
+This model is the XLM model trained on Wikipedia text in 100 languages. The preprocessing included tokenization with byte-pair-encoding. See the [GitHub repo](https://github.com/facebookresearch/XLM#the-17-and-100-languages) and the [associated paper](https://arxiv.org/pdf/1911.02116.pdf) for further details on the training data and training procedure.
+
+[Conneau et al. (2020)](https://arxiv.org/pdf/1911.02116.pdf) report that this model has 16 layers, 1280 hidden states, 16 attention heads, and the dimension of the feed-forward layer is 1520. The vocabulary size is 200k and the total number of parameters is 570M (see Table 7).
+ 
+# Evaluation
+
+## Testing Data, Factors & Metrics
+
+The model developers evaluated the model on the XNLI cross-lingual classification task (see the [XNLI data card](https://huggingface.co/datasets/xnli) for more details on XNLI) using the metric of test accuracy. See the [GitHub Repo](https://arxiv.org/pdf/1911.02116.pdf) for further details on the testing data, factors and metrics.
+
+## Results
+
+For xlm-mlm-100-1280, the test accuracy on the XNLI cross-lingual classification task in English (en), Spanish (es), German (de), Arabic (ar), Chinese (zh) and Urdu (ur) are:
+
+|Language| en | es  | de | ar | zh | ur |
+|:------:|:--:|:---:|:--:|:--:|:--:|:--:|
+|        |83.7|76.6 |73.6|67.4|71.7|62.9|
+
+See the [GitHub repo](https://github.com/facebookresearch/XLM#ii-cross-lingual-language-model-pretraining-xlm) for further details.
+
+# Environmental Impact
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** More information needed
+- **Hours used:** More information needed
+- **Cloud Provider:** More information needed
+- **Compute Region:** More information needed
+- **Carbon Emitted:** More information needed
+
+# Technical Specifications
+
+[Conneau et al. (2020)](https://arxiv.org/pdf/1911.02116.pdf) report that this model has 16 layers, 1280 hidden states, 16 attention heads, and the dimension of the feed-forward layer is 1520. The vocabulary size is 200k and the total number of parameters is 570M (see Table 7).
+
+# Citation
+
+**BibTeX:**
+
+```bibtex
+@article{lample2019cross,
+  title={Cross-lingual language model pretraining},
+  author={Lample, Guillaume and Conneau, Alexis},
+  journal={arXiv preprint arXiv:1901.07291},
+  year={2019}
+}
+```
+
+**APA:**
+- Lample, G., & Conneau, A. (2019). Cross-lingual language model pretraining. arXiv preprint arXiv:1901.07291.
+
+# Model Card Authors 
+
+This model card was written by the team at Hugging Face.
+
+# How to Get Started with the Model
+
+More information needed. See the [ipython notebook](https://github.com/facebookresearch/XLM/blob/main/generate-embeddings.ipynb) in the associated [GitHub repo](https://github.com/facebookresearch/XLM#the-17-and-100-languages) for examples.