MugheesAwan11/bge-base-securiti-dataset-1-v19

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 tokens
• Similarity Function: Cosine Similarity
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("MugheesAwan11/bge-base-securiti-dataset-1-v19")
# Run inference
sentences = [
    'Consent of an individual is valid if it is reasonable to expect that an individual to whom the organization’s activities are directed would understand the nature, purpose, and consequences of the collection, use, or disclosure of the personal information to which they are consenting. The information must be provided in manageable and easily accessible ways to data subjects and data subjects must be allowed to withdraw consent. If there is a use or disclosure a data subject would not reasonably expect to be occurring, such as certain sharing of information with a third party or the tracking of location, express consent would likely be required. However, the data subject’s consent may not be required for certain data processing activities such as when the collection is “clearly” in the interests of the individual and consent cannot be obtained in a timely way, data is being collected in the course of employment, journalistic, is already publicly available, information is being collected for the detection and prevention of fraud or for',
    'How should information be provided to data subjects in manageable and easily accessible ways?',
    'Which state, following California, Virginia, and Colorado, recently passed privacy legislation like the VCDPA?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Dataset: dim_768
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.4021
cosine_accuracy@3	0.5567
cosine_accuracy@5	0.6804
cosine_accuracy@10	0.7526
cosine_precision@1	0.4021
cosine_precision@3	0.1856
cosine_precision@5	0.1361
cosine_precision@10	0.0753
cosine_recall@1	0.4021
cosine_recall@3	0.5567
cosine_recall@5	0.6804
cosine_recall@10	0.7526
cosine_ndcg@10	0.565
cosine_mrr@10	0.506
cosine_map@100	0.5167

Information Retrieval

• Dataset: dim_512
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.3918
cosine_accuracy@3	0.5876
cosine_accuracy@5	0.6289
cosine_accuracy@10	0.7526
cosine_precision@1	0.3918
cosine_precision@3	0.1959
cosine_precision@5	0.1258
cosine_precision@10	0.0753
cosine_recall@1	0.3918
cosine_recall@3	0.5876
cosine_recall@5	0.6289
cosine_recall@10	0.7526
cosine_ndcg@10	0.5625
cosine_mrr@10	0.5031
cosine_map@100	0.5142

Information Retrieval

• Dataset: dim_256
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.3814
cosine_accuracy@3	0.5773
cosine_accuracy@5	0.6392
cosine_accuracy@10	0.7113
cosine_precision@1	0.3814
cosine_precision@3	0.1924
cosine_precision@5	0.1278
cosine_precision@10	0.0711
cosine_recall@1	0.3814
cosine_recall@3	0.5773
cosine_recall@5	0.6392
cosine_recall@10	0.7113
cosine_ndcg@10	0.5461
cosine_mrr@10	0.4931
cosine_map@100	0.5068

Training Details

Training Dataset

Unnamed Dataset

• Size: 882 training samples
• Columns: positive and anchor
• Approximate statistics based on the first 1000 samples:

  	positive	anchor
  type	string	string
  details	min: 18 tokens mean: 227.32 tokens max: 414 tokens	min: 10 tokens mean: 21.98 tokens max: 102 tokens

• Samples:

  positive	anchor
  Leader in Data Privacy View Events Spotlight Talks Education Contact Us Schedule a Demo Products By Use Cases By Roles Data Command Center View Learn more Asset and Data Discovery Discover dark and native data assets Learn more Data Access Intelligence & Governance Identify which users have access to sensitive data and prevent unauthorized access Learn more Data Privacy Automation PrivacyCenter.Cloud	Data Mapping
  data subject must be notified of any such extension within one month of receiving the request, along with the reasons for the delay and the possibility of complaining to the supervisory authority. The right to restrict processing applies when the data subject contests data accuracy, the processing is unlawful, and the data subject opposes erasure and requests restriction. The controller must inform data subjects before any such restriction is lifted. Under GDPR, the data subject also has the right to obtain from the controller the rectification of inaccurate personal data and to have incomplete personal data completed. Article: 22 Under PDPL, if a decision is based solely on automated processing of personal data intended to assess the data subject regarding his/her performance at work, financial standing, credit-worthiness, reliability, or conduct, then the data subject has the right to request processing in a manner that is not solely automated. This right shall not apply where the decision is taken in the course of entering into	What is the requirement for notifying the data subject of any extension under GDPR and PDPL?
  Automation PrivacyCenter.Cloud	Data Mapping

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256
      ],
      "matryoshka_weights": [
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 4
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: True
• load_best_model_at_end: True
• optim: adamw_torch_fused
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_256_cosine_map@100	dim_512_cosine_map@100	dim_768_cosine_map@100
0.3571	10	4.0517	-	-	-
0.7143	20	2.5778	-	-	-
1.0	28	-	0.5304	0.5224	0.5234
1.0714	30	2.1161	-	-	-
1.4286	40	1.5394	-	-	-
1.7857	50	1.5162	-	-	-
2.0	56	-	0.5412	0.5382	0.5185
2.1429	60	1.202	-	-	-
2.5	70	1.0456	-	-	-
2.8571	80	1.1341	-	-	-
3.0	84	-	0.5340	0.5554	0.5498
3.2143	90	0.8724	-	-	-
3.5714	100	0.932	-	-	-
3.9286	110	0.9548	-	-	-
4.0	112	-	0.5296	0.5487	0.5491
0.3571	10	0.9958	-	-	-
0.7143	20	0.8264	-	-	-
1.0	28	-	0.5155	0.5250	0.5269
1.0714	30	0.7969	-	-	-
1.4286	40	0.6244	-	-	-
1.7857	50	0.6368	-	-	-
2.0	56	-	0.5034	0.5314	0.5233
2.1429	60	0.4748	-	-	-
2.5	70	0.4037	-	-	-
2.8571	80	0.4615	-	-	-
3.0	84	-	0.5079	0.5145	0.5155
3.2143	90	0.3148	-	-	-
3.5714	100	0.4142	-	-	-
3.9286	110	0.366	-	-	-
4.0	112	-	0.5068	0.5142	0.5167

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.14
• Sentence Transformers: 3.0.1
• Transformers: 4.41.2
• PyTorch: 2.1.2+cu121
• Accelerate: 0.31.0
• Datasets: 2.19.1
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}