finetuned_arctic / README.md
acpotts's picture
Add new SentenceTransformer model.
2d373f7 verified
|
raw
history blame
31.9 kB
metadata
base_model: Snowflake/snowflake-arctic-embed-m
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
  - dot_accuracy@1
  - dot_accuracy@3
  - dot_accuracy@5
  - dot_accuracy@10
  - dot_precision@1
  - dot_precision@3
  - dot_precision@5
  - dot_precision@10
  - dot_recall@1
  - dot_recall@3
  - dot_recall@5
  - dot_recall@10
  - dot_ndcg@10
  - dot_mrr@10
  - dot_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:800
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >-
      What is the importance of having a human fallback system in automated
      systems, especially for the American public?
    sentences:
      - |-
        ing a system from use. Automated systems should not be designed 
        with an intent or reasonably foreseeable possibility of endangering 
        your safety or the safety of your community. They should be designed 
        to proactively protect you from harms stemming from unintended, 
        yet foreseeable, uses or impacts of automated systems. You should be 
        protected from inappropriate or irrelevant data use in the design, de­
        velopment, and deployment of automated systems, and from the 
        compounded harm of its reuse. Independent evaluation and report­
        ing that confirms that the system is safe and effective, including re­
        porting of steps taken to mitigate potential harms, should be per­
        formed and the results made public whenever possible. 
        15
      - >-
        with disabilities. 

        In addition to being able to opt out and use a human alternative, the
        American public deserves a human fallback 

        system in the event that an automated system fails or causes harm. No
        matter how rigorously an automated system is 

        tested, there will always be situations for which the system fails. The
        American public deserves protection via human 

        review against these outlying or unexpected scenarios. In the case of
        time-critical systems, the public should not have 

        to wait—immediate human consideration and fallback should be available.
        In many time-critical systems, such a 

        remedy is already immediately available, such as a building manager who
        can open a door in the case an automated 

        card access system fails.
      - >-
        information used to build or validate the risk assessment shall be open
        to public inspection," and that assertions 

        of trade secrets cannot be used "to quash discovery in a criminal matter
        by a party to a criminal case." 

        22
  - source_sentence: >-
      What type of information is required to be open to public inspection in
      relation to risk assessment?
    sentences:
      - >-
        HOW THESE PRINCIPLES CAN MOVE INTO PRACTICE

        Real-life examples of how these principles can become reality, through
        laws, policies, and practical 

        technical and sociotechnical approaches to protecting rights,
        opportunities, and access. 

        The federal government is working to combat discrimination in mortgage
        lending. The Depart­

        ment of Justice has launched a nationwide initiative to combat
        redlining, which includes reviewing how 

        lenders who may be avoiding serving communities of color are conducting
        targeted marketing and advertising.51 

        This initiative will draw upon strong partnerships across federal
        agencies, including the Consumer Financial
      - >-
        reuse 

        Relevant and high-quality data. Data used as part of any automated
        system’s creation, evaluation, or 

        deployment should be relevant, of high quality, and tailored to the task
        at hand. Relevancy should be 

        established based on research-backed demonstration of the causal
        influence of the data to the specific use case 

        or justified more generally based on a reasonable expectation of
        usefulness in the domain and/or for the 

        system design or ongoing development. Relevance of data should not be
        established solely by appealing to 

        its historical connection to the outcome. High quality and tailored data
        should be representative of the task at
      - >-
        information used to build or validate the risk assessment shall be open
        to public inspection," and that assertions 

        of trade secrets cannot be used "to quash discovery in a criminal matter
        by a party to a criminal case." 

        22
  - source_sentence: >-
      Who is the Senior Policy Advisor for Data and Democracy at the White House
      Office of Science and Technology Policy?
    sentences:
      - >-
        products, advanced platforms and services, “Internet of Things” (IoT)
        devices, and smart city products and 

        services. 

        Welcome:

        

        Rashida Richardson, Senior Policy Advisor for Data and Democracy, White
        House Office of Science and

        Technology Policy

        

        Karen Kornbluh, Senior Fellow and Director of the Digital Innovation and
        Democracy Initiative, German

        Marshall Fund

        Moderator: 

        Devin E. Willis, Attorney, Division of Privacy and Identity Protection,
        Bureau of Consumer Protection, Federal 

        Trade Commission 

        Panelists: 

        

        Tamika L. Butler, Principal, Tamika L. Butler Consulting

        

        Jennifer Clark, Professor and Head of City and Regional Planning,
        Knowlton School of Engineering, Ohio

        State University

        
      - >-
        ENDNOTES

        35. Carrie Johnson. Flaws plague a tool meant to help low-risk federal
        prisoners win early release. NPR.

        Jan. 26, 2022.
        https://www.npr.org/2022/01/26/1075509175/flaws-plague-a-tool-meant-to-help-low­

        risk-federal-prisoners-win-early-release.; Carrie Johnson. Justice
        Department works to curb racial bias

        in deciding who's released from prison. NPR. Apr. 19, 2022. https://

        www.npr.org/2022/04/19/1093538706/justice-department-works-to-curb-racial-bias-in-deciding­

        whos-released-from-pris; National Institute of Justice. 2021 Review and
        Revalidation of the First Step Act

        Risk Assessment Tool. National Institute of Justice NCJ 303859. Dec.,
        2021. https://www.ojp.gov/

        pdffiles1/nij/303859.pdf
      - >-
        https://themarkup.org/machine-learning/2022/01/11/this-private-equity-firm-is-amassing-companies­

        that-collect-data-on-americas-children

        77. Reed Albergotti. Every employee who leaves Apple becomes an
        ‘associate’: In job databases used by

        employers to verify resume information, every former Apple employee’s
        title gets erased and replaced with

        a generic title. The Washington Post. Feb. 10, 2022.

        https://www.washingtonpost.com/technology/2022/02/10/apple-associate/

        78. National Institute of Standards and Technology. Privacy Framework
        Perspectives and Success

        Stories. Accessed May 2, 2022.

        https://www.nist.gov/privacy-framework/getting-started-0/perspectives-and-success-stories
  - source_sentence: >-
      What actions has the Consumer Financial Protection Bureau taken regarding
      black-box credit models?
    sentences:
      - >-
        under-ecoa-fcra/

        91. Federal Trade Commission. Using Consumer Reports for Credit
        Decisions: What to Know About

        Adverse Action and Risk-Based Pricing Notices. Accessed May 2, 2022.

        https://www.ftc.gov/business-guidance/resources/using-consumer-reports-credit-decisions-what­

        know-about-adverse-action-risk-based-pricing-notices#risk

        92. Consumer Financial Protection Bureau. CFPB Acts to Protect the
        Public from Black-Box Credit

        Models Using Complex Algorithms. May 26, 2022.

        https://www.consumerfinance.gov/about-us/newsroom/cfpb-acts-to-protect-the-public-from-black­

        box-credit-models-using-complex-algorithms/

        93. Anthony Zaller. California Passes Law Regulating Quotas In
        Warehouses  What Employers Need to
      - >-
        https://www.nytimes.com/2020/12/29/technology/facial-recognition-misidentify-jail.html;
        Khari

        Johnson. How Wrongful Arrests Based on AI Derailed 3 Men's Lives. Wired.
        Mar. 7, 2022. https://

        www.wired.com/story/wrongful-arrests-ai-derailed-3-mens-lives/

        32. Student Borrower Protection Center. Educational Redlining. Student
        Borrower Protection Center

        Report. Feb. 2020.
        https://protectborrowers.org/wp-content/uploads/2020/02/Education-Redlining­

        Report.pdf

        33. Jeffrey Dastin. Amazon scraps secret AI recruiting tool that showed
        bias against women. Reuters. Oct.

        10, 2018.
        https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps­

        secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G
      - >-
        including automated tenant background screening and facial
        recognition-based controls to enter or exit 

        housing complexes. Employment-related concerning uses included
        discrimination in automated hiring 

        screening and workplace surveillance. Various panelists raised the
        limitations of existing privacy law as a key 

        concern, pointing out that students should be able to reinvent
        themselves and require privacy of their student 

        records and education-related data in order to do so. The overarching
        concerns of surveillance in these 

        domains included concerns about the chilling effects of surveillance on
        student expression, inappropriate
  - source_sentence: >-
      What percentage of racy results did Google cut for searches like 'Latina
      teenager' in March 2022?
    sentences:
      - >-
        they've used drugs, or whether they've expressed interest in LGBTQI+
        groups, and then use that data to 

        forecast student success.76 Parents and education experts have expressed
        concern about collection of such

        sensitive data without express parental consent, the lack of
        transparency in how such data is being used, and

        the potential for resulting discriminatory impacts.

         Many employers transfer employee data to third party job verification
        services. This information is then used

        by potential future employers, banks, or landlords. In one case, a
        former employee alleged that a

        company supplied false data about her job title which resulted in a job
        offer being revoked.77

        37
      - >-
        Software Discriminates Against Disabled Students. Center for Democracy
        and Technology. Nov. 16, 2020.

        https://cdt.org/insights/how-automated-test-proctoring-software-discriminates-against-disabled­

        students/

        46. Ziad Obermeyer, et al., Dissecting racial bias in an algorithm used
        to manage the health of

        populations, 366 Science (2019),
        https://www.science.org/doi/10.1126/science.aax2342.

        66
      - >-
        2022.
        https://www.reuters.com/technology/google-cuts-racy-results-by-30-searches-like-latina­

        teenager-2022-03-30/

        40. Safiya Umoja Noble. Algorithms of Oppression: How Search Engines
        Reinforce Racism. NYU Press.

        Feb. 2018. https://nyupress.org/9781479837243/algorithms-of-oppression/

        41. Paresh Dave. Google cuts racy results by 30% for searches like
        'Latina teenager'. Reuters. Mar. 30,

        2022.
        https://www.reuters.com/technology/google-cuts-racy-results-by-30-searches-like-latina­

        teenager-2022-03-30/

        42. Miranda Bogen. All the Ways Hiring Algorithms Can Introduce Bias.
        Harvard Business Review. May

        6, 2019.
        https://hbr.org/2019/05/all-the-ways-hiring-algorithms-can-introduce-bias
model-index:
  - name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: Unknown
          type: unknown
        metrics:
          - type: cosine_accuracy@1
            value: 0.815
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.935
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.95
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.965
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.815
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.31166666666666665
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.19
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09649999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.815
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.935
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.95
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.965
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8954135083695783
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8723333333333333
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8741632101558571
            name: Cosine Map@100
          - type: dot_accuracy@1
            value: 0.815
            name: Dot Accuracy@1
          - type: dot_accuracy@3
            value: 0.935
            name: Dot Accuracy@3
          - type: dot_accuracy@5
            value: 0.95
            name: Dot Accuracy@5
          - type: dot_accuracy@10
            value: 0.965
            name: Dot Accuracy@10
          - type: dot_precision@1
            value: 0.815
            name: Dot Precision@1
          - type: dot_precision@3
            value: 0.31166666666666665
            name: Dot Precision@3
          - type: dot_precision@5
            value: 0.19
            name: Dot Precision@5
          - type: dot_precision@10
            value: 0.09649999999999999
            name: Dot Precision@10
          - type: dot_recall@1
            value: 0.815
            name: Dot Recall@1
          - type: dot_recall@3
            value: 0.935
            name: Dot Recall@3
          - type: dot_recall@5
            value: 0.95
            name: Dot Recall@5
          - type: dot_recall@10
            value: 0.965
            name: Dot Recall@10
          - type: dot_ndcg@10
            value: 0.8954135083695783
            name: Dot Ndcg@10
          - type: dot_mrr@10
            value: 0.8723333333333333
            name: Dot Mrr@10
          - type: dot_map@100
            value: 0.8741632101558571
            name: Dot Map@100

SentenceTransformer based on Snowflake/snowflake-arctic-embed-m

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: Snowflake/snowflake-arctic-embed-m
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 768 tokens
  • Similarity Function: Cosine Similarity

Model Sources

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("acpotts/finetuned_arctic")
# Run inference
sentences = [
    "What percentage of racy results did Google cut for searches like 'Latina teenager' in March 2022?",
    "2022. https://www.reuters.com/technology/google-cuts-racy-results-by-30-searches-like-latina\xad\nteenager-2022-03-30/\n40. Safiya Umoja Noble. Algorithms of Oppression: How Search Engines Reinforce Racism. NYU Press.\nFeb. 2018. https://nyupress.org/9781479837243/algorithms-of-oppression/\n41. Paresh Dave. Google cuts racy results by 30% for searches like 'Latina teenager'. Reuters. Mar. 30,\n2022. https://www.reuters.com/technology/google-cuts-racy-results-by-30-searches-like-latina\xad\nteenager-2022-03-30/\n42. Miranda Bogen. All the Ways Hiring Algorithms Can Introduce Bias. Harvard Business Review. May\n6, 2019. https://hbr.org/2019/05/all-the-ways-hiring-algorithms-can-introduce-bias",
    "they've used drugs, or whether they've expressed interest in LGBTQI+ groups, and then use that data to \nforecast student success.76 Parents and education experts have expressed concern about collection of such\nsensitive data without express parental consent, the lack of transparency in how such data is being used, and\nthe potential for resulting discriminatory impacts.\n• Many employers transfer employee data to third party job verification services. This information is then used\nby potential future employers, banks, or landlords. In one case, a former employee alleged that a\ncompany supplied false data about her job title which resulted in a job offer being revoked.77\n37",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Metric Value
cosine_accuracy@1 0.815
cosine_accuracy@3 0.935
cosine_accuracy@5 0.95
cosine_accuracy@10 0.965
cosine_precision@1 0.815
cosine_precision@3 0.3117
cosine_precision@5 0.19
cosine_precision@10 0.0965
cosine_recall@1 0.815
cosine_recall@3 0.935
cosine_recall@5 0.95
cosine_recall@10 0.965
cosine_ndcg@10 0.8954
cosine_mrr@10 0.8723
cosine_map@100 0.8742
dot_accuracy@1 0.815
dot_accuracy@3 0.935
dot_accuracy@5 0.95
dot_accuracy@10 0.965
dot_precision@1 0.815
dot_precision@3 0.3117
dot_precision@5 0.19
dot_precision@10 0.0965
dot_recall@1 0.815
dot_recall@3 0.935
dot_recall@5 0.95
dot_recall@10 0.965
dot_ndcg@10 0.8954
dot_mrr@10 0.8723
dot_map@100 0.8742

Training Details

Training Dataset

Unnamed Dataset

  • Size: 800 training samples
  • Columns: sentence_0 and sentence_1
  • Approximate statistics based on the first 800 samples:
    sentence_0 sentence_1
    type string string
    details
    • min: 11 tokens
    • mean: 20.11 tokens
    • max: 36 tokens
    • min: 3 tokens
    • mean: 127.42 tokens
    • max: 512 tokens
  • Samples:
    sentence_0 sentence_1
    What are some of the principles proposed for the ethical use of AI and automated systems? lems with legislation, and some courts extending longstanding statutory protections to new and emerging tech­
    nologies. There are companies working to incorporate additional protections in their design and use of auto­
    mated systems, and researchers developing innovative guardrails. Advocates, researchers, and government
    organizations have proposed principles for the ethical use of AI and other automated systems. These include
    the Organization for Economic Co-operation and Development’s (OECD’s) 2019 Recommendation on Artificial
    Intelligence, which includes principles for responsible stewardship of trustworthy AI and which the United
    How are companies and researchers addressing the challenges posed by new and emerging technologies in relation to legislation? lems with legislation, and some courts extending longstanding statutory protections to new and emerging tech­
    nologies. There are companies working to incorporate additional protections in their design and use of auto­
    mated systems, and researchers developing innovative guardrails. Advocates, researchers, and government
    organizations have proposed principles for the ethical use of AI and other automated systems. These include
    the Organization for Economic Co-operation and Development’s (OECD’s) 2019 Recommendation on Artificial
    Intelligence, which includes principles for responsible stewardship of trustworthy AI and which the United
    What is the purpose of reporting summary information about automated systems in plain language? any operators or others who need to understand the system, and calibrated to the level of risk based on the
    context. Reporting that includes summary information about these automated systems in plain language and
    assessments of the clarity and quality of the notice and explanations should be made public whenever possible.
    6
  • Loss: MatryoshkaLoss with these parameters:
    {
        "loss": "MultipleNegativesRankingLoss",
        "matryoshka_dims": [
            768,
            512,
            256,
            128,
            64
        ],
        "matryoshka_weights": [
            1,
            1,
            1,
            1,
            1
        ],
        "n_dims_per_step": -1
    }
    

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: steps
  • per_device_train_batch_size: 20
  • per_device_eval_batch_size: 20
  • num_train_epochs: 5
  • multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: steps
  • prediction_loss_only: True
  • per_device_train_batch_size: 20
  • per_device_eval_batch_size: 20
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 5e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1
  • num_train_epochs: 5
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.0
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: False
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: False
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • eval_use_gather_object: False
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: round_robin

Training Logs

Epoch Step cosine_map@100
1.0 40 0.8676
1.25 50 0.8670
2.0 80 0.8731
2.5 100 0.8722
1.0 40 0.8641
1.25 50 0.8654
2.0 80 0.8674
2.5 100 0.8706
3.0 120 0.8659
3.75 150 0.8697
4.0 160 0.8706
5.0 200 0.8742

Framework Versions

  • Python: 3.10.12
  • Sentence Transformers: 3.1.1
  • Transformers: 4.44.2
  • PyTorch: 2.4.1+cu121
  • Accelerate: 0.34.2
  • Datasets: 3.0.0
  • Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}