Add SetFit model

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

1_Pooling/config.json

@@ -1,7 +1,7 @@
 {
-  "word_embedding_dimension": 768,
-  "pooling_mode_cls_token": false,
-  "pooling_mode_mean_tokens": true,
+  "word_embedding_dimension": 1024,
+  "pooling_mode_cls_token": true,
+  "pooling_mode_mean_tokens": false,
   "pooling_mode_max_tokens": false,
   "pooling_mode_mean_sqrt_len_tokens": false,
   "pooling_mode_weightedmean_tokens": false,

README.md

@@ -5,26 +5,24 @@ tags:
 - sentence-transformers
 - text-classification
 - generated_from_setfit_trainer
-base_model: sentence-transformers/all-mpnet-base-v2
+base_model: BAAI/bge-m3
 metrics:
 - accuracy
 widget:
-- text: "What is the difference between ACID compliance and BASE topology in distributed\
-    \ database systems?\n               While designing a high throughput RDBMS like\
-    \ Amazon Aurora, how would you choose between consistency model that is highly\
-    \ optimistic and one that highly paranoid in conflictedupdate scenarios"
-- text: What is the primary function of the Apache Hive metastore in a Hadoop ecosystem,
-    and how does it differ from a traditional relational database management system?
-- text: What is the primary purpose of employing Entity-Controlled Vocabulary (ESS)
-    in open data publishing, according to industry experts at Microsoft?
-- text: How do organizations prioritize innovation to strive in a rapidly changing
-    industry landscape driven by technological and societal shifts?
-- text: How do societal norms influence the emergence of new business models in unstable
-    economies?
+- text: What is the primary difference between a Bayesian neural network and a traditional
+    feedforward neural network in the context of machine learning?
+- text: What is the difference betweensupervised and unsupervised machine learning
+    algorithms in terms of data labeling and model training?
+- text: What is the primary application of Natural Language Processing (NLP) in Google's
+    BERT language model, and how does it utilize masked language modeling to improve
+    contextual understanding?
+- text: What is the main advantage of using GraphQL over traditional RESTful APIs,
+    as demonstrated by social media giant Facebook in their Facebook ADS API?
+- text: Qui est Robin Mancini ?
 pipeline_tag: text-classification
 inference: true
 model-index:
-- name: SetFit with sentence-transformers/all-mpnet-base-v2
+- name: SetFit with BAAI/bge-m3
   results:
   - task:
       type: text-classification
@@ -39,9 +37,9 @@ model-index:
       name: Accuracy
 ---
 
-# SetFit with sentence-transformers/all-mpnet-base-v2
+# SetFit with BAAI/bge-m3
 
-This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
 
 The model has been trained using an efficient few-shot learning technique that involves:
 
@@ -52,9 +50,9 @@ The model has been trained using an efficient few-shot learning technique that i
 
 ### Model Description
 - **Model Type:** SetFit
-- **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
+- **Sentence Transformer body:** [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3)
 - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
-- **Maximum Sequence Length:** 384 tokens
+- **Maximum Sequence Length:** 8192 tokens
 - **Number of Classes:** 2 classes
 <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
 <!-- - **Language:** Unknown -->
@@ -67,10 +65,10 @@ The model has been trained using an efficient few-shot learning technique that i
 - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
 
 ### Model Labels
-| Label    | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
-|:---------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| semantic | <ul><li>'How do artificial intelligence systems navigate the trade-off between simplicity and accuracy when modeling complex real-world phenomena?'</li><li>'How do complex systems, consisting of many interconnected components, give rise to emergent properties that cannot be predicted from the characteristics of their individual parts?'</li><li>'How do complex systems, such as those found in nature and human societies, exhibit emergent properties that arise from the interactions of individual components?'</li></ul> |
-| lexical  | <ul><li>'What is the primary difference between a generative adversarial network (GAN) and a variational autoencoder (VAE) in deep learning?'</li><li>'What is the primary difference between a Decision Tree and a Random Forest in Machine Learning, and how do they alleviate overfitting?'</li><li>'What is the primary difference between a Bayesian neural network and a traditional feedforward neural network in the context of machine learning?'</li></ul>                                                                    |
+| Label    | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+|:---------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| lexical  | <ul><li>'What is the definition of semantics in the context ofontology-based data integration, and how does it differ from outright data normalization, as implementented in graph databases like neo4j orAmazon Neptune?'</li><li>'What is the primary application of graph convolutional neural networks (GCNNs) in natural language processing (NLP) for modeling syntactic dependencies in parsing?'</li><li>"What is the distinguising feature of Apache Hive's Metadata Tables, used for maintaining and managingtables in Hadoop Distributed File System (HDFS)?"</li></ul> |
+| semantic | <ul><li>'What is a key challenge faced by managers in sustaining a work culture that encourages creativity, innovation, and critical thinking within the technological industry globally?'</li><li>'How might shifting societal values influence the dynamics between multinational corporations and governments, leading to Changes in the global economic landscape?'</li><li>'How does the allocation of limited resources affect the allocation of decision-making power within an organization?'</li></ul>                                                                    |
 
 ## Evaluation
 
@@ -97,7 +95,7 @@ from setfit import SetFitModel
 # Download from the 🤗 Hub
 model = SetFitModel.from_pretrained("yaniseuranova/setfit-paraphrase-mpnet-base-v2-sst2")
 # Run inference
-preds = model("How do societal norms influence the emergence of new business models in unstable economies?")
+preds = model("Qui est Robin Mancini ?")
 ```
 
 <!--
@@ -129,12 +127,12 @@ preds = model("How do societal norms influence the emergence of new business mod
 ### Training Set Metrics
 | Training set | Min | Median  | Max |
 |:-------------|:----|:--------|:----|
-| Word count   | 4   | 18.6566 | 56  |
+| Word count   | 4   | 19.1392 | 56  |
 
 | Label    | Training Sample Count |
 |:---------|:----------------------|
-| lexical  | 47                    |
-| semantic | 52                    |
+| lexical  | 36                    |
+| semantic | 43                    |
 
 ### Training Hyperparameters
 - batch_size: (16, 16)
@@ -156,36 +154,27 @@ preds = model("How do societal norms influence the emergence of new business mod
 ### Training Results
 | Epoch   | Step    | Training Loss | Validation Loss |
 |:-------:|:-------:|:-------------:|:---------------:|
-| 0.0032  | 1       | 0.3177        | -               |
-| 0.1592  | 50      | 0.0905        | -               |
-| 0.3185  | 100     | 0.0013        | -               |
-| 0.4777  | 150     | 0.0011        | -               |
-| 0.6369  | 200     | 0.0002        | -               |
-| 0.7962  | 250     | 0.0003        | -               |
-| 0.9554  | 300     | 0.0001        | -               |
-| 1.0     | 314     | -             | 0.0001          |
-| 1.1146  | 350     | 0.0001        | -               |
-| 1.2739  | 400     | 0.0001        | -               |
-| 1.4331  | 450     | 0.0001        | -               |
-| 1.5924  | 500     | 0.0001        | -               |
-| 1.7516  | 550     | 0.0001        | -               |
-| 1.9108  | 600     | 0.0001        | -               |
-| 2.0     | 628     | -             | 0.0             |
-| 2.0701  | 650     | 0.0001        | -               |
-| 2.2293  | 700     | 0.0           | -               |
-| 2.3885  | 750     | 0.0001        | -               |
-| 2.5478  | 800     | 0.0           | -               |
-| 2.7070  | 850     | 0.0001        | -               |
-| 2.8662  | 900     | 0.0001        | -               |
-| **3.0** | **942** | **-**         | **0.0**         |
-| 3.0255  | 950     | 0.0001        | -               |
-| 3.1847  | 1000    | 0.0           | -               |
-| 3.3439  | 1050    | 0.0001        | -               |
-| 3.5032  | 1100    | 0.0001        | -               |
-| 3.6624  | 1150    | 0.0           | -               |
-| 3.8217  | 1200    | 0.0001        | -               |
-| 3.9809  | 1250    | 0.0001        | -               |
-| 4.0     | 1256    | -             | 0.0             |
+| 0.0050  | 1       | 0.1549        | -               |
+| 0.2475  | 50      | 0.0045        | -               |
+| 0.4950  | 100     | 0.0009        | -               |
+| 0.7426  | 150     | 0.0005        | -               |
+| 0.9901  | 200     | 0.0005        | -               |
+| 1.0     | 202     | -             | 0.0001          |
+| 1.2376  | 250     | 0.0006        | -               |
+| 1.4851  | 300     | 0.0006        | -               |
+| 1.7327  | 350     | 0.0005        | -               |
+| 1.9802  | 400     | 0.0004        | -               |
+| 2.0     | 404     | -             | 0.0             |
+| 2.2277  | 450     | 0.0003        | -               |
+| 2.4752  | 500     | 0.0003        | -               |
+| 2.7228  | 550     | 0.0003        | -               |
+| 2.9703  | 600     | 0.0003        | -               |
+| **3.0** | **606** | **-**         | **0.0**         |
+| 3.2178  | 650     | 0.0003        | -               |
+| 3.4653  | 700     | 0.0004        | -               |
+| 3.7129  | 750     | 0.0003        | -               |
+| 3.9604  | 800     | 0.0002        | -               |
+| 4.0     | 808     | -             | 0.0             |
 
 * The bold row denotes the saved checkpoint.
 ### Framework Versions

config.json

@@ -1,24 +1,28 @@
 {
-  "_name_or_path": "checkpoints/step_942",
+  "_name_or_path": "checkpoints/step_606",
   "architectures": [
-    "MPNetModel"
+    "XLMRobertaModel"
   ],
   "attention_probs_dropout_prob": 0.1,
   "bos_token_id": 0,
+  "classifier_dropout": null,
   "eos_token_id": 2,
   "hidden_act": "gelu",
   "hidden_dropout_prob": 0.1,
-  "hidden_size": 768,
+  "hidden_size": 1024,
   "initializer_range": 0.02,
-  "intermediate_size": 3072,
+  "intermediate_size": 4096,
   "layer_norm_eps": 1e-05,
-  "max_position_embeddings": 514,
-  "model_type": "mpnet",
-  "num_attention_heads": 12,
-  "num_hidden_layers": 12,
+  "max_position_embeddings": 8194,
+  "model_type": "xlm-roberta",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "output_past": true,
   "pad_token_id": 1,
-  "relative_attention_num_buckets": 32,
+  "position_embedding_type": "absolute",
   "torch_dtype": "float32",
   "transformers_version": "4.39.0",
-  "vocab_size": 30527
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 250002
 }

config_sentence_transformers.json

@@ -1,8 +1,8 @@
 {
   "__version__": {
-    "sentence_transformers": "2.0.0",
-    "transformers": "4.6.1",
-    "pytorch": "1.8.1"
+    "sentence_transformers": "2.2.2",
+    "transformers": "4.33.0",
+    "pytorch": "2.1.2+cu121"
   },
   "prompts": {},
   "default_prompt_name": null

config_setfit.json

@@ -1,7 +1,7 @@
 {
+  "normalize_embeddings": false,
   "labels": [
     "lexical",
     "semantic"
-  ],
-  "normalize_embeddings": false
+  ]
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dbf6d3930a01c83d13c8e9917fadace89ffe82b20e0e8e1e79879f17c3f3ffea
-size 437967672
+oid sha256:b6b631443242ee9cb7eaa44b335a5b8a0932d0f7730c1e523a2972f095dd5fe6
+size 2271064456

model_head.pkl

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3eb7a8f7e177ed92e49ddc0fd6d56cff9748420121ded82c0713ee52aa1e2294
-size 7039
+oid sha256:8f26505603a392dfebb8ca914a16aa7e94aeeb8b35f89376e80a56616a8b08a4
+size 9087

sentence_bert_config.json

@@ -1,4 +1,4 @@
 {
-  "max_seq_length": 384,
+  "max_seq_length": 8192,
   "do_lower_case": false
 }

special_tokens_map.json

@@ -42,7 +42,7 @@
     "single_word": false
   },
   "unk_token": {
-    "content": "[UNK]",
+    "content": "<unk>",
     "lstrip": false,
     "normalized": false,
     "rstrip": false,

tokenizer_config.json

@@ -27,20 +27,12 @@
     "3": {
       "content": "<unk>",
       "lstrip": false,
-      "normalized": true,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "104": {
-      "content": "[UNK]",
-      "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "single_word": false,
       "special": true
     },
-    "30526": {
+    "250001": {
       "content": "<mask>",
       "lstrip": true,
       "normalized": false,
@@ -52,21 +44,19 @@
   "bos_token": "<s>",
   "clean_up_tokenization_spaces": true,
   "cls_token": "<s>",
-  "do_lower_case": true,
   "eos_token": "</s>",
   "mask_token": "<mask>",
-  "max_length": 128,
-  "model_max_length": 512,
+  "max_length": 8192,
+  "model_max_length": 8192,
   "pad_to_multiple_of": null,
   "pad_token": "<pad>",
   "pad_token_type_id": 0,
   "padding_side": "right",
   "sep_token": "</s>",
+  "sp_model_kwargs": {},
   "stride": 0,
-  "strip_accents": null,
-  "tokenize_chinese_chars": true,
-  "tokenizer_class": "MPNetTokenizer",
+  "tokenizer_class": "XLMRobertaTokenizer",
   "truncation_side": "right",
   "truncation_strategy": "longest_first",
-  "unk_token": "[UNK]"
+  "unk_token": "<unk>"
 }