File size: 25,788 Bytes
c3eb07b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 |
---
language: []
library_name: sentence-transformers
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- dataset_size:1M<n<10M
- loss:TripletLoss
base_model: sentence-transformers/paraphrase-MiniLM-L12-v2
metrics:
- cosine_accuracy
- dot_accuracy
- manhattan_accuracy
- euclidean_accuracy
- max_accuracy
widget:
- source_sentence: 'method: Making reflective work practices visible'
sentences:
- 'method: Job quality takes into account both wage and non-wage attributes of a
job.'
- 'purpose: There could therefore be rank differences in the leadership behavioural
patterns of managers.'
- 'negative: SN has a positive effect on the user''s intention to use toward the
SNS.'
- source_sentence: 'findings: Proposed logistics framework'
sentences:
- 'purpose: However these may not be the only reasons for undertaking collection
evaluation.'
- 'purpose: Clearly, there is variation in the definition and understanding of the
term sustainability.'
- 'purpose: The study is based on a panel data regression analysis of 234 SMEs over
a 10-year period (2004-2013).'
- source_sentence: 'method: Electoral campaigns and party websites'
sentences:
- 'method: Track, leadership style, and team outcomes'
- 'purpose: , three CKM strategies that organizations use to manage customer knowledge
are considered.'
- 'findings: Motherhood directly affects career progression.'
- source_sentence: 'negative: Entrepreneurship education in Iran'
sentences:
- 'negative: Sensemaking as local weather'
- 'findings: In the next section, we will develop hypotheses to explain retail banner
divestment timing.'
- 'negative: Thus, the purpose of this paper is to review AR in retailing within
business-oriented research.'
- source_sentence: 'purpose: 2.2 Decentralization and participation'
sentences:
- 'purpose: Social norm approach and feedback'
- 'findings: The upper path of the model represents how counter-knowledge directly
affects ACAP, reducing HC.'
- 'purpose: Online strategy building requires a series of steps.'
pipeline_tag: sentence-similarity
model-index:
- name: SentenceTransformer based on sentence-transformers/paraphrase-MiniLM-L12-v2
results:
- task:
type: triplet
name: Triplet
dataset:
name: triplet
type: triplet
metrics:
- type: cosine_accuracy
value: 0.6998206089274619
name: Cosine Accuracy
- type: dot_accuracy
value: 0.39671483834759774
name: Dot Accuracy
- type: manhattan_accuracy
value: 0.6998506744703453
name: Manhattan Accuracy
- type: euclidean_accuracy
value: 0.7153344290553406
name: Euclidean Accuracy
- type: max_accuracy
value: 0.7153344290553406
name: Max Accuracy
---
# SentenceTransformer based on sentence-transformers/paraphrase-MiniLM-L12-v2
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/paraphrase-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L12-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/paraphrase-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L12-v2) <!-- at revision 3ab2765205fa23269bcc8c8e08ae5b1c35203ab4 -->
- **Maximum Sequence Length:** 256 tokens
- **Output Dimensionality:** 384 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("gubartz/facet_retriever")
# Run inference
sentences = [
'purpose: 2.2 Decentralization and participation',
'purpose: Social norm approach and feedback',
'findings: The upper path of the model represents how counter-knowledge directly affects ACAP, reducing HC.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
## Evaluation
### Metrics
#### Triplet
* Dataset: `triplet`
* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| **cosine_accuracy** | **0.6998** |
| dot_accuracy | 0.3967 |
| manhattan_accuracy | 0.6999 |
| euclidean_accuracy | 0.7153 |
| max_accuracy | 0.7153 |
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 1,541,116 training samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 1000 samples:
| | anchor | positive | negative |
|:--------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string | string |
| details | <ul><li>min: 9 tokens</li><li>mean: 42.16 tokens</li><li>max: 187 tokens</li></ul> | <ul><li>min: 10 tokens</li><li>mean: 42.77 tokens</li><li>max: 183 tokens</li></ul> | <ul><li>min: 8 tokens</li><li>mean: 38.65 tokens</li><li>max: 227 tokens</li></ul> |
* Samples:
| anchor | positive | negative |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>purpose: study attempts to fill this gap by examining firm-specific capabilities of Turkish outward FDI firms.</code> | <code>purpose: In short, the above-mentioned percentages show the lack of usage of knowledge sharing and collaborative technologies in some research institutions in Malaysia due to perceived causes such as non-availability of technology, lack of support, absent of teamwork culture, and lack of knowledge and training.</code> | <code>purpose: While SMA alone must not be used to gather and analyze these voices, these tools can guide organizations in relating to their publics, increasing the way groups identify with them and motivating these groups to enter into relationships with them.</code> |
| <code>purpose: In this section of the paper, we try to explain citizen attitudes towards sustainable procurement.</code> | <code>purpose: Different from previous studies to concern key factors for motivating consumers' online buying behavior and behavioral intention (Liang and Lim, 2011; Zhang et al., 2013), such finding add knowledge in the filed by finding the meaningful affective mechanism of consumers in OFGB.</code> | <code>purpose: Task significance is not significantly different among generational cohorts of knowledge workers.</code> |
| <code>purpose: However, the extensive use of information technology (IT) also comes with related security problems caused by the abstract nature of interacting systems - technical and organizational - and the seemingly lack of or inferior control of data or information.</code> | <code>purpose: No previous research using cluster analysis in nursing homes was found, but clusters identified in this study are lower than in previous hospital-based research into patients experiences and satisfaction used as cluster variables (Grondahl et al., 2011).</code> | <code>purpose: Yet, this engagement has tended to only involve a small section of the overall medical workforce in practice, raising questions about the nature of medical engagement more broadly and the mechanisms needed to enhance these processes.</code> |
* Loss: [<code>TripletLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#tripletloss) with these parameters:
```json
{
"distance_metric": "TripletDistanceMetric.EUCLIDEAN",
"triplet_margin": 5
}
```
### Evaluation Dataset
#### Unnamed Dataset
* Size: 199,564 evaluation samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 1000 samples:
| | anchor | positive | negative |
|:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string | string |
| details | <ul><li>min: 9 tokens</li><li>mean: 42.64 tokens</li><li>max: 165 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 42.42 tokens</li><li>max: 197 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 38.23 tokens</li><li>max: 193 tokens</li></ul> |
* Samples:
| anchor | positive | negative |
|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>purpose: However, it seems obvious that, in the long run, Green OA can be seen as leading progressively to the disappearance of the "traditional" publication model and, possibly, of scientific publishers altogether unless they reconsider their business model and adapt to the new situation.</code> | <code>purpose: Considering the transcendence of the sustainable development agenda in the UDRD, it was decided to search for explicit references to the issue of risk in the proposed indicators, finding a correspondence between four indicators of the development agenda and indicators proposed for the implementation of the Sendai Framework (Maskrey, 2016).</code> | <code>purpose: Finally, the terms of the permanent multinomial corresponding to the particular manufacturing system may be listed and the resulting graphs may be obtained and used for structurally analyzing the capabilities of the manufacturing system in different areas.</code> |
| <code>purpose: To what extent do information science and the other disciplines demonstrate interest in social network theory and social network analysis?RQ2.</code> | <code>purpose: This study explores relationships between relationship commitment, cooperative behavior and alliance performance from the perspectives of both companies and contract farmers.</code> | <code>purpose: 4.1 The respondents' health literacy skills</code> |
| <code>purpose: The evidence discussed above shows the nature of forecasting connections in the income growth across the globe.</code> | <code>purpose: Namely, the paper confirms that there is vast deviation between the European countries when it comes to consumer trust in banking in general but also related to each studied banking service.</code> | <code>purpose: Healthcare is one of the major sectors in which Lean production is being considered and adopted as an improvement program (Poksinska, 2010).</code> |
* Loss: [<code>TripletLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#tripletloss) with these parameters:
```json
{
"distance_metric": "TripletDistanceMetric.EUCLIDEAN",
"triplet_margin": 5
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: epoch
- `per_device_train_batch_size`: 64
- `per_device_eval_batch_size`: 128
- `gradient_accumulation_steps`: 16
- `num_train_epochs`: 5
- `warmup_ratio`: 0.1
- `fp16`: True
- `load_best_model_at_end`: True
- `auto_find_batch_size`: True
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 64
- `per_device_eval_batch_size`: 128
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 5
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: True
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional
</details>
### Training Logs
| Epoch | Step | Training Loss | loss | triplet_cosine_accuracy |
|:-------:|:--------:|:-------------:|:----------:|:-----------------------:|
| 0.3322 | 500 | 4.2859 | - | - |
| 0.6645 | 1000 | 3.693 | - | - |
| 0.9967 | 1500 | 3.5602 | - | - |
| 1.0 | 1505 | - | 3.4908 | 0.6914 |
| 1.3289 | 2000 | 3.427 | - | - |
| 1.6611 | 2500 | 3.3854 | - | - |
| 1.9934 | 3000 | 3.3551 | - | - |
| 2.0 | 3010 | - | 3.3604 | 0.7000 |
| 2.3256 | 3500 | 3.2353 | - | - |
| 2.6578 | 4000 | 3.221 | - | - |
| 2.9900 | 4500 | 3.2038 | - | - |
| **3.0** | **4515** | **-** | **3.3203** | **0.7026** |
| 3.3223 | 5000 | 3.1019 | - | - |
| 3.6545 | 5500 | 3.0942 | - | - |
| 3.9867 | 6000 | 3.085 | - | - |
| 4.0 | 6020 | - | 3.3177 | 0.7014 |
| 4.3189 | 6500 | 3.0129 | - | - |
| 4.6512 | 7000 | 3.0083 | - | - |
| 4.9834 | 7500 | 2.9971 | - | - |
| 5.0 | 7525 | - | 3.3264 | 0.6998 |
* The bold row denotes the saved checkpoint.
### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.0.0
- Transformers: 4.41.0
- PyTorch: 2.3.0+cu121
- Accelerate: 0.30.1
- Datasets: 2.19.1
- Tokenizers: 0.19.1
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### TripletLoss
```bibtex
@misc{hermans2017defense,
title={In Defense of the Triplet Loss for Person Re-Identification},
author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
year={2017},
eprint={1703.07737},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--> |