README.md

---
base_model: Snowflake/snowflake-arctic-embed-m
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
- dot_accuracy@1
- dot_accuracy@3
- dot_accuracy@5
- dot_accuracy@10
- dot_precision@1
- dot_precision@3
- dot_precision@5
- dot_precision@10
- dot_recall@1
- dot_recall@3
- dot_recall@5
- dot_recall@10
- dot_ndcg@10
- dot_mrr@10
- dot_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:600
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: What are the potential risks associated with the impersonation
    and cyber-attacks mentioned in the context?
  sentences:
  - "Technology Engagement Center \nUber Technologies \nUniversity of Pittsburgh \n\
    Undergraduate Student \nCollaborative \nUpturn \nUS Technology Policy Committee\
    \ \nof the Association of Computing \nMachinery \nVirginia Puccio \nVisar Berisha\
    \ and Julie Liss \nXR Association \nXR Safety Initiative \n• As an additional\
    \ effort to reach out to stakeholders regarding the RFI, OSTP conducted two listening\
    \ sessions\nfor members of the public. The listening sessions together drew upwards\
    \ of 300 participants. The Science and\nTechnology Policy Institute produced a\
    \ synopsis of both the RFI submissions and the feedback at the listening\nsessions.115\n\
    61"
  - "across all subgroups, which could leave the groups facing underperformance with\
    \ worse outcomes than \nif no GAI system were used. Disparate or reduced performance\
    \ for lower-resource languages also \npresents challenges to model adoption, inclusion,\
    \ and accessibility, and may make preservation of \nendangered languages more\
    \ difficult if GAI systems become embedded in everyday processes that would \notherwise\
    \ have been opportunities to use these languages.  \nBias is mutually reinforcing\
    \ with the problem of undesired homogenization, in which GAI systems \nproduce\
    \ skewed distributions of outputs that are overly uniform (for example, repetitive\
    \ aesthetic styles"
  - "impersonation, cyber-attacks, and weapons creation. \nCBRN Information or Capabilities;\
    \ \nInformation Security \nMS-2.6-007 Regularly evaluate GAI system vulnerabilities\
    \ to possible circumvention of safety \nmeasures.  \nCBRN Information or Capabilities;\
    \ \nInformation Security \nAI Actor Tasks: AI Deployment, AI Impact Assessment,\
    \ Domain Experts, Operation and Monitoring, TEVV"
- source_sentence: What techniques are suggested to assess and manage statistical
    biases related to GAI content provenance?
  sentences:
  - "2 \nThis work was informed by public feedback and consultations with diverse\
    \ stakeholder groups as part of NIST’s \nGenerative AI Public Working Group (GAI\
    \ PWG). The GAI PWG was an open, transparent, and collaborative \nprocess, facilitated\
    \ via a virtual workspace, to obtain multistakeholder input on GAI risk management\
    \ and to \ninform NIST’s approach. \nThe focus of the GAI PWG was limited to four\
    \ primary considerations relevant to GAI: Governance, Content \nProvenance, Pre-deployment\
    \ Testing, and Incident Disclosure (further described in Appendix A). As such,\
    \ the \nsuggested actions in this document primarily address these considerations.\
    \ \nFuture revisions of this profile will include additional AI RMF subcategories,\
    \ risks, and suggested actions based \non additional considerations of GAI as\
    \ the space evolves and empirical evidence indicates additional risks. A \nglossary\
    \ of terms pertinent to GAI risk management will be developed and hosted on NIST’s\
    \ Trustworthy &"
  - "30 \nMEASURE 2.2: Evaluations involving human subjects meet applicable requirements\
    \ (including human subject protection) and are \nrepresentative of the relevant\
    \ population. \nAction ID \nSuggested Action \nGAI Risks \nMS-2.2-001 Assess and\
    \ manage statistical biases related to GAI content provenance through \ntechniques\
    \ such as re-sampling, re-weighting, or adversarial training. \nInformation Integrity;\
    \ Information \nSecurity; Harmful Bias and \nHomogenization \nMS-2.2-002 \nDocument\
    \ how content provenance data is tracked and how that data interacts \nwith privacy\
    \ and security. Consider: Anonymizing data to protect the privacy of \nhuman subjects;\
    \ Leveraging privacy output filters; Removing any personally \nidentifiable information\
    \ (PII) to prevent potential harm or misuse. \nData Privacy; Human AI \nConfiguration;\
    \ Information \nIntegrity; Information Security; \nDangerous, Violent, or Hateful\
    \ \nContent \nMS-2.2-003 Provide human subjects with options to withdraw participation\
    \ or revoke their"
  - "humans (e.g., intelligence tests, professional licensing exams) does not guarantee\
    \ GAI system validity or \nreliability in those domains. Similarly, jailbreaking\
    \ or prompt engineering tests may not systematically \nassess validity or reliability\
    \ risks.  \nMeasurement gaps can arise from mismatches between laboratory and\
    \ real-world settings. Current \ntesting approaches often remain focused on laboratory\
    \ conditions or restricted to benchmark test \ndatasets and in silico techniques\
    \ that may not extrapolate well to—or directly assess GAI impacts in real-\nworld\
    \ conditions. For example, current measurement gaps for GAI make it difficult to\
    \ precisely estimate \nits potential ecosystem-level or longitudinal risks and\
    \ related political, social, and economic impacts. \nGaps between benchmarks and\
    \ real-world use of GAI systems may likely be exacerbated due to prompt \nsensitivity\
    \ and broad heterogeneity of contexts of use. \nA.1.5. Structured Public Feedback"
- source_sentence: How does the absence of an explanation regarding data usage affect
    parents' ability to contest decisions made in child maltreatment assessments?
  sentences:
  - '62. See, e.g., Federal Trade Commission. Data Brokers: A Call for Transparency
    and Accountability. May

    2014.

    https://www.ftc.gov/system/files/documents/reports/data-brokers-call-transparency-accountability­

    report-federal-trade-commission-may-2014/140527databrokerreport.pdf; Cathy O’Neil.

    Weapons of Math Destruction. Penguin Books. 2017.

    https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction

    63. See, e.g., Rachel Levinson-Waldman, Harsha Pandurnga, and Faiza Patel. Social
    Media Surveillance by

    the U.S. Government. Brennan Center for Justice. Jan. 7, 2022.

    https://www.brennancenter.org/our-work/research-reports/social-media-surveillance-us-government;

    Shoshana Zuboff. The Age of Surveillance Capitalism: The Fight for a Human Future
    at the New Frontier of

    Power. Public Affairs. 2019.

    64. Angela Chen. Why the Future of Life Insurance May Depend on Your Online Presence.
    The Verge. Feb.

    7, 2019.'
  - "NOTICE & \nEXPLANATION \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides\
    \ a brief summary of the problems which the principle seeks to address and protect\
    \ \nagainst, including illustrative examples. \nAutomated systems now determine\
    \ opportunities, from employment to credit, and directly shape the American \n\
    public’s experiences, from the courtroom to online classrooms, in ways that profoundly\
    \ impact people’s lives. But this \nexpansive impact is not always visible. An\
    \ applicant might not know whether a person rejected their resume or a \nhiring\
    \ algorithm moved them to the bottom of the list. A defendant in the courtroom\
    \ might not know if a judge deny­\ning their bail is informed by an automated\
    \ system that labeled them “high risk.” From correcting errors to contesting \n\
    decisions, people are often denied the knowledge they need to address the impact\
    \ of automated systems on their lives."
  - 'ever being notified that data was being collected and used as part of an algorithmic
    child maltreatment

    risk assessment.84 The lack of notice or an explanation makes it harder for those
    performing child

    maltreatment assessments to validate the risk assessment and denies parents knowledge
    that could help them

    contest a decision.

    41'
- source_sentence: How should automated systems be tested to ensure they are free
    from algorithmic discrimination?
  sentences:
  - "Homogenization? arXiv. https://arxiv.org/pdf/2211.13972 \nBoyarskaya, M. et al.\
    \ (2020) Overcoming Failures of Imagination in AI Infused System Development and\
    \ \nDeployment. arXiv. https://arxiv.org/pdf/2011.13416 \nBrowne, D. et al. (2023)\
    \ Securing the AI Pipeline. Mandiant. \nhttps://www.mandiant.com/resources/blog/securing-ai-pipeline\
    \ \nBurgess, M. (2024) Generative AI’s Biggest Security Flaw Is Not Easy to Fix.\
    \ WIRED. \nhttps://www.wired.com/story/generative-ai-prompt-injection-hacking/\
    \ \nBurtell, M. et al. (2024) The Surprising Power of Next Word Prediction: Large\
    \ Language Models \nExplained, Part 1. Georgetown Center for Security and Emerging\
    \ Technology. \nhttps://cset.georgetown.edu/article/the-surprising-power-of-next-word-prediction-large-language-\n\
    models-explained-part-1/ \nCanadian Centre for Cyber Security (2023) Generative\
    \ artificial intelligence (AI) - ITSAP.00.041. \nhttps://www.cyber.gc.ca/en/guidance/generative-artificial-intelligence-ai-itsap00041"
  - "relevant biological and chemical threat knowledge and information is often publicly\
    \ accessible, LLMs \ncould facilitate its analysis or synthesis, particularly\
    \ by individuals without formal scientific training or \nexpertise.  \nRecent research\
    \ on this topic found that LLM outputs regarding biological threat creation and\
    \ attack \nplanning provided minimal assistance beyond traditional search engine\
    \ queries, suggesting that state-of-\nthe-art LLMs at the time these studies were\
    \ conducted do not substantially increase the operational \nlikelihood of such\
    \ an attack. The physical synthesis development, production, and use of chemical\
    \ or \nbiological agents will continue to require both applicable expertise and\
    \ supporting materials and \ninfrastructure. The impact of GAI on chemical or\
    \ biological agent misuse will depend on what the key \nbarriers for malicious\
    \ actors are (e.g., whether information access is one such barrier), and how well\
    \ GAI \ncan help actors address those barriers."
  - "WHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated\
    \ systems are meant to serve as a blueprint for the development of additional\
    \ \ntechnical standards and practices that are tailored for particular sectors\
    \ and contexts. \nAny automated system should be tested to help ensure it is free\
    \ from algorithmic discrimination before it can be \nsold or used. Protection\
    \ against algorithmic discrimination should include designing to ensure equity,\
    \ broadly \nconstrued.  Some algorithmic discrimination is already prohibited\
    \ under existing anti-discrimination law. The \nexpectations set out below describe\
    \ proactive technical and policy steps that can be taken to not only \nreinforce\
    \ those legal protections but extend beyond them to ensure equity for underserved\
    \ communities48 \neven in circumstances where a specific legal protection may\
    \ not be clearly established. These protections"
- source_sentence: What rights do applicants have if their application for credit
    is denied according to the CFPB?
  sentences:
  - "listed organizations and individuals:\nAccenture \nAccess Now \nACT | The App\
    \ Association \nAHIP \nAIethicist.org \nAirlines for America \nAlliance for Automotive\
    \ Innovation \nAmelia Winger-Bearskin \nAmerican Civil Liberties Union \nAmerican\
    \ Civil Liberties Union of \nMassachusetts \nAmerican Medical Association \nARTICLE19\
    \ \nAttorneys General of the District of \nColumbia, Illinois, Maryland, \nMichigan,\
    \ Minnesota, New York, \nNorth Carolina, Oregon, Vermont, \nand Washington \n\
    Avanade \nAware \nBarbara Evans \nBetter Identity Coalition \nBipartisan Policy\
    \ Center \nBrandon L. Garrett and Cynthia \nRudin \nBrian Krupp \nBrooklyn Defender\
    \ Services \nBSA | The Software Alliance \nCarnegie Mellon University \nCenter\
    \ for Democracy & \nTechnology \nCenter for New Democratic \nProcesses \nCenter\
    \ for Research and Education \non Accessible Technology and \nExperiences at University\
    \ of \nWashington, Devva Kasnitz, L Jean \nCamp, Jonathan Lazar, Harry \nHochheiser\
    \ \nCenter on Privacy & Technology at \nGeorgetown Law \nCisco Systems"
  - "even if the inferences are not accurate (e.g., confabulations), and especially\
    \ if they reveal information \nthat the individual considers sensitive or that\
    \ is used to disadvantage or harm them. \nBeyond harms from information exposure\
    \ (such as extortion or dignitary harm), wrong or inappropriate \ninferences of\
    \ PII can contribute to downstream or secondary harmful impacts. For example,\
    \ predictive \ninferences made by GAI models based on PII or protected attributes\
    \ can contribute to adverse decisions, \nleading to representational or allocative\
    \ harms to individuals or groups (see Harmful Bias and \nHomogenization below)."
  - "information in their credit report.\" The CFPB has also asserted that \"[t]he\
    \ law gives every applicant the right to \na specific explanation if their application\
    \ for credit was denied, and that right is not diminished simply because \na company\
    \ uses a complex algorithm that it doesn't understand.\"92 Such explanations illustrate\
    \ a shared value \nthat certain decisions need to be explained. \nA California\
    \ law requires that warehouse employees are provided with notice and explana-\n\
    tion about quotas, potentially facilitated by automated systems, that apply to\
    \ them. Warehous-\ning employers in California that use quota systems (often facilitated\
    \ by algorithmic monitoring systems) are \nrequired to provide employees with\
    \ a written description of each quota that applies to the employee, including\
    \ \n“quantified number of tasks to be performed or materials to be produced or\
    \ handled, within the defined"
model-index:
- name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: Unknown
      type: unknown
    metrics:
    - type: cosine_accuracy@1
      value: 0.98
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 1.0
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 1.0
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 1.0
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.98
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.3333333333333334
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.19999999999999996
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09999999999999998
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.98
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 1.0
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 1.0
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 1.0
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.9913092975357145
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.9883333333333333
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.9883333333333334
      name: Cosine Map@100
    - type: dot_accuracy@1
      value: 0.98
      name: Dot Accuracy@1
    - type: dot_accuracy@3
      value: 1.0
      name: Dot Accuracy@3
    - type: dot_accuracy@5
      value: 1.0
      name: Dot Accuracy@5
    - type: dot_accuracy@10
      value: 1.0
      name: Dot Accuracy@10
    - type: dot_precision@1
      value: 0.98
      name: Dot Precision@1
    - type: dot_precision@3
      value: 0.3333333333333334
      name: Dot Precision@3
    - type: dot_precision@5
      value: 0.19999999999999996
      name: Dot Precision@5
    - type: dot_precision@10
      value: 0.09999999999999998
      name: Dot Precision@10
    - type: dot_recall@1
      value: 0.98
      name: Dot Recall@1
    - type: dot_recall@3
      value: 1.0
      name: Dot Recall@3
    - type: dot_recall@5
      value: 1.0
      name: Dot Recall@5
    - type: dot_recall@10
      value: 1.0
      name: Dot Recall@10
    - type: dot_ndcg@10
      value: 0.9913092975357145
      name: Dot Ndcg@10
    - type: dot_mrr@10
      value: 0.9883333333333333
      name: Dot Mrr@10
    - type: dot_map@100
      value: 0.9883333333333334
      name: Dot Map@100
---

# SentenceTransformer based on Snowflake/snowflake-arctic-embed-m

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m) <!-- at revision e2b128b9fa60c82b4585512b33e1544224ffff42 -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("vincha77/finetuned_arctic")
# Run inference
sentences = [
    'What rights do applicants have if their application for credit is denied according to the CFPB?',
    'information in their credit report." The CFPB has also asserted that "[t]he law gives every applicant the right to \na specific explanation if their application for credit was denied, and that right is not diminished simply because \na company uses a complex algorithm that it doesn\'t understand."92 Such explanations illustrate a shared value \nthat certain decisions need to be explained. \nA California law requires that warehouse employees are provided with notice and explana-\ntion about quotas, potentially facilitated by automated systems, that apply to them. Warehous-\ning employers in California that use quota systems (often facilitated by algorithmic monitoring systems) are \nrequired to provide employees with a written description of each quota that applies to the employee, including \n“quantified number of tasks to be performed or materials to be produced or handled, within the defined',
    'even if the inferences are not accurate (e.g., confabulations), and especially if they reveal information \nthat the individual considers sensitive or that is used to disadvantage or harm them. \nBeyond harms from information exposure (such as extortion or dignitary harm), wrong or inappropriate \ninferences of PII can contribute to downstream or secondary harmful impacts. For example, predictive \ninferences made by GAI models based on PII or protected attributes can contribute to adverse decisions, \nleading to representational or allocative harms to individuals or groups (see Harmful Bias and \nHomogenization below).',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.98       |
| cosine_accuracy@3   | 1.0        |
| cosine_accuracy@5   | 1.0        |
| cosine_accuracy@10  | 1.0        |
| cosine_precision@1  | 0.98       |
| cosine_precision@3  | 0.3333     |
| cosine_precision@5  | 0.2        |
| cosine_precision@10 | 0.1        |
| cosine_recall@1     | 0.98       |
| cosine_recall@3     | 1.0        |
| cosine_recall@5     | 1.0        |
| cosine_recall@10    | 1.0        |
| cosine_ndcg@10      | 0.9913     |
| cosine_mrr@10       | 0.9883     |
| **cosine_map@100**  | **0.9883** |
| dot_accuracy@1      | 0.98       |
| dot_accuracy@3      | 1.0        |
| dot_accuracy@5      | 1.0        |
| dot_accuracy@10     | 1.0        |
| dot_precision@1     | 0.98       |
| dot_precision@3     | 0.3333     |
| dot_precision@5     | 0.2        |
| dot_precision@10    | 0.1        |
| dot_recall@1        | 0.98       |
| dot_recall@3        | 1.0        |
| dot_recall@5        | 1.0        |
| dot_recall@10       | 1.0        |
| dot_ndcg@10         | 0.9913     |
| dot_mrr@10          | 0.9883     |
| dot_map@100         | 0.9883     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 600 training samples
* Columns: <code>sentence_0</code> and <code>sentence_1</code>
* Approximate statistics based on the first 600 samples:
  |         | sentence_0                                                                         | sentence_1                                                                           |
  |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                               |
  | details | <ul><li>min: 12 tokens</li><li>mean: 21.21 tokens</li><li>max: 39 tokens</li></ul> | <ul><li>min: 21 tokens</li><li>mean: 182.02 tokens</li><li>max: 512 tokens</li></ul> |
* Samples:
  | sentence_0                                                                                                                                 | sentence_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
  |:-------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>What are the responsibilities of AI Actors in monitoring reported issues related to GAI system performance?</code>                   | <code>45 <br>MG-4.1-007 <br>Verify that AI Actors responsible for monitoring reported issues can effectively <br>evaluate GAI system performance including the application of content <br>provenance data tracking techniques, and promptly escalate issues for response. <br>Human-AI Configuration; <br>Information Integrity <br>AI Actor Tasks: AI Deployment, Affected Individuals and Communities, Domain Experts, End-Users, Human Factors, Operation and <br>Monitoring <br> <br>MANAGE 4.2: Measurable activities for continual improvements are integrated into AI system updates and include regular <br>engagement with interested parties, including relevant AI Actors. <br>Action ID <br>Suggested Action <br>GAI Risks <br>MG-4.2-001 Conduct regular monitoring of GAI systems and publish reports detailing the <br>performance, feedback received, and improvements made. <br>Harmful Bias and Homogenization <br>MG-4.2-002 <br>Practice and follow incident response plans for addressing the generation of</code> |
  | <code>How are measurable activities for continual improvements integrated into AI system updates according to the context provided?</code> | <code>45 <br>MG-4.1-007 <br>Verify that AI Actors responsible for monitoring reported issues can effectively <br>evaluate GAI system performance including the application of content <br>provenance data tracking techniques, and promptly escalate issues for response. <br>Human-AI Configuration; <br>Information Integrity <br>AI Actor Tasks: AI Deployment, Affected Individuals and Communities, Domain Experts, End-Users, Human Factors, Operation and <br>Monitoring <br> <br>MANAGE 4.2: Measurable activities for continual improvements are integrated into AI system updates and include regular <br>engagement with interested parties, including relevant AI Actors. <br>Action ID <br>Suggested Action <br>GAI Risks <br>MG-4.2-001 Conduct regular monitoring of GAI systems and publish reports detailing the <br>performance, feedback received, and improvements made. <br>Harmful Bias and Homogenization <br>MG-4.2-002 <br>Practice and follow incident response plans for addressing the generation of</code> |
  | <code>What is the main function of the app discussed in Samantha Cole's article from June 26, 2019?</code>                                 | <code>them<br>10. Samantha Cole. This Horrifying App Undresses a Photo of Any Woman With a Single Click. Motherboard.<br>June 26, 2019. https://www.vice.com/en/article/kzm59x/deepnude-app-creates-fake-nudes-of-any-woman<br>11. Lauren Kaori Gurley. Amazon’s AI Cameras Are Punishing Drivers for Mistakes They Didn’t Make.<br>Motherboard. Sep. 20, 2021. https://www.vice.com/en/article/88npjv/amazons-ai-cameras-are-punishing­<br>drivers-for-mistakes-they-didnt-make<br>63</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `num_train_epochs`: 5
- `multi_dataset_batch_sampler`: round_robin

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 5
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin

</details>

### Training Logs
| Epoch  | Step | cosine_map@100 |
|:------:|:----:|:--------------:|
| 1.0    | 38   | 0.965          |
| 1.3158 | 50   | 0.9783         |
| 2.0    | 76   | 0.9767         |
| 2.6316 | 100  | 0.9833         |
| 3.0    | 114  | 0.9883         |


### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.1.1
- Transformers: 4.44.2
- PyTorch: 2.4.1+cu121
- Accelerate: 0.34.2
- Datasets: 3.0.0
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->