Update README.md (#7)

- Update README.md (9fa15f967dfb5513d806e2c444ee5ac7141870a7)


Co-authored-by: bob chesebrough <[email protected]>

README.md

@@ -4,6 +4,9 @@ language:
 license: apache-2.0
 tags:
 - generated_from_trainer
+- bert-base-uncased
+- text-classification
+- fp32 
 datasets:
 - glue
 metrics:
@@ -72,10 +75,10 @@ should probably proofread and complete it, then remove this comment. -->
 
 # bert-base-uncased-mrpc
 
-This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on the **GLUE MRPC dataset**.
+This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on the **GLUE MRPC dataset**. The GLUE MRPC dataset, from The [Microsoft Research Paraphrase Corpus (Dolan & Brockett, 2005)](https://www.tensorflow.org/datasets/catalog/glue) is a corpus of sentence pairs automatically extracted from online news sources, with human annotations for whether the sentences in the pair are semantically equivalent.
 
-It is a pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. 
-This model is uncased: it does not make a difference between **"english"** and **"English"**.
+It is a pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805). 
+This model, bert-base-uncased-mrpc, is uncased: it does not make a difference between **"english"** and **"English"**. Masked language modeling predicts a masked token in a sequence, and the model can attend to tokens bidirectionally. This means the model has full access to the tokens on the left and right. Masked language modeling is great for tasks that require a good contextual understanding of an entire sequence. BERT is an example of a masked language model. For this model, you don’t need labels (also known as an unsupervised task) because the next word (MLM) is the label
 BERT base model (uncased)
 
 It provides:
@@ -107,18 +110,26 @@ The following hyperparameters were used during training:
 - Datasets 1.14.0
 - Tokenizers 0.11.6
 
-- # To use:
+# To use:
 ```python
 from transformers import BertTokenizer, BertModel
 tokenizer = BertTokenizer.from_pretrained('Intel/bert-base-uncased-mrpc')
 model = BertModel.from_pretrained("Intel/bert-base-uncased-mrpc")
-# text = "according to the theory of aerodynamics and wind tunnel experiments the bumble bee is unable to fly. This is bcause the size, weight, and shape of his body in relation to total wingspread makes flying impossible. But, the bumble bee being ignorant of these pround scientific truths goes ahead and flies anyway, and manages to make a little honey everyday."
 text = "The inspector analyzed the soundness in the building."
 encoded_input = tokenizer(text, return_tensors='pt')
 output = model(**encoded_input)
 # print BaseModelOutputWithPoolingAndCrossAttentions and  pooler_output
-# output similar to:
+
+# Print tokens * ids in of inmput string below
+print('Tokenized Text: ', tokenizer.tokenize(text), '\n')
+print('Token IDs: ', tokenizer.convert_tokens_to_ids(tokenizer.tokenize(text)))
+
+#Print tokens in text
+encoded_input['input_ids'][0]
+tokenizer.convert_ids_to_tokens(encoded_input['input_ids'][0])
 ```
+# Output similar to:
+```python
 BaseModelOutputWithPoolingAndCrossAttentions(last_hidden_state=tensor([[[ 0.0219,  0.1258, -0.8529,  ...,  0.6416,  0.6275,  0.5583],
          [ 0.3125, -0.1921, -0.9895,  ...,  0.6069,  1.8431, -0.5939],
          [ 0.6147, -0.6098, -0.3517,  ..., -0.1145,  1.1748, -0.7104],
@@ -130,14 +141,45 @@ BaseModelOutputWithPoolingAndCrossAttentions(last_hidden_state=tensor([[[ 0.0219
          -0.9176, -0.9994,  0.2962,  0.2891, -0.3301,  0.8786,  0.9234, -0.7643,
           0.2487, -0.5245, -0.0649, -0.6722,  0.8550,  1.0000, -0.7785,  0.5322,
           0.6056,  0.4622,  0.2838,  0.5501,  0.6981,  0.2597, -0.7896, -0.1189,
+```
 
-```python
-# Print tokens * ids in of inmput string below
-print('Tokenized Text: ', tokenizer.tokenize(text), '\n')
-print('Token IDs: ', tokenizer.convert_tokens_to_ids(tokenizer.tokenize(text)))
+# Related work on QuantizationAwareTraining
+An Int8 Quantized version of this model can be found [link](https://huggingface.co/Intel/bert-base-uncased-mrpc-int8-qat-inc)
 
-#Print tokens in text
-encoded_input['input_ids'][0]
-tokenizer.convert_ids_to_tokens(encoded_input['input_ids'][0])
-```
-          
+This is an INT8 PyTorch model quantized with huggingface/optimum-intel through the usage of Intel® Neural Compressor.
+
+# Ethical Considerations and Limitations
+bert-base-uncased-mrpc can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
+
+Therefore, before deploying any applications of bert-base-uncased-mrpc, developers should perform safety testing.
+
+# Caveats and Recommendations
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
+
+Here are a couple of useful links to learn more about Intel's AI software:
+
+- Intel Neural Compressor [link](https://github.com/intel/neural-compressor)
+- Intel Extension for Transformers [link](https://github.com/intel/intel-extension-for-transformers)
+
+# Disclaimer
+The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please cosult an attorney before using this model for commercial purposes.
+
+# BibTeX entry and citation info
+```bibtex
+@article{DBLP:journals/corr/abs-1810-04805,
+  author    = {Jacob Devlin and
+               Ming{-}Wei Chang and
+               Kenton Lee and
+               Kristina Toutanova},
+  title     = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language
+               Understanding},
+  journal   = {CoRR},
+  volume    = {abs/1810.04805},
+  year      = {2018},
+  url       = {http://arxiv.org/abs/1810.04805},
+  archivePrefix = {arXiv},
+  eprint    = {1810.04805},
+  timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
+  biburl    = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},
+  bibsource = {dblp computer science bibliography, https://dblp.org}
+}