metadata

base_model: BAAI/bge-base-en-v1.5
datasets: []
language:
  - en
library_name: sentence-transformers
license: apache-2.0
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:6300
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: Consumer Products segment decreased 10% to $3,572.5 million.
    sentences:
      - >-
        What was the impact of the Federal Reserve’s policy changes on Schwab
        money market funds in 2022?
      - >-
        What was the total revenue of Hasbro's Consumer Products segment in
        2022?
      - >-
        How much did the company's currently payable U.S. taxes amount to in
        2023?
  - source_sentence: >-
      PricewaterhouseCoopers LLP is mentioned as the Firm’s independent
      registered public accounting firm (PCAOB ID 238) in the audit of the
      Consolidated Financial Statements.
    sentences:
      - >-
        Where in the document can the Consolidated Financial Statements be found
        as mentioned in a 2024 report?
      - >-
        What type of firm is PricewaterhouseCoopers LLP as mentioned in the
        context of auditing?
      - Which note in the report provides details about legal proceedings?
  - source_sentence: >-
      If, in the future, foreign exchange or capital control restrictions were
      to be imposed and become applicable to us, such restrictions could
      potentially reduce the amounts that we would be able to receive from our
      Macao, Hong Kong and mainland China subsidiaries.
    sentences:
      - >-
        What are the potential consequences for the parent company if foreign
        exchange or capital control restrictions were imposed in the future?
      - What is described under Item 8 in the context of a financial document?
      - >-
        What types of investments are primarily included in the Goldman Sachs'
        investments in funds at NAV as of December 2023?
  - source_sentence: >-
      Determining income tax provisions involves forecasting future financial
      results, planning potential tax strategies, and evaluating the probability
      of sustaining tax positions against audits.
    sentences:
      - What type of company is Johnson & Johnson described as?
      - >-
        What determines the fair value of available-for-sale short-term
        investments?
      - >-
        What factors influence the determination of income tax provisions and
        related tax balances?
  - source_sentence: >-
      During the fiscal year ended March 31, 2023, a $118 million tax charge
      increased the valuation allowance on Swiss deferred tax assets, leading to
      a higher effective tax rate.
    sentences:
      - >-
        What accounted for the significant tax rate increase in fiscal year
        2023?
      - >-
        What percentage of the box office revenue in the U.S./Canada was
        generated by the three largest exhibitors in 2023?
      - >-
        What percentage of eBay's 2023 net revenues were attributed to
        international markets?

BGE base Financial Matryoshka

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-base-en-v1.5
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ValentinaKim/bge-base-financial-matryoshka4")
# Run inference
sentences = [
    'During the fiscal year ended March 31, 2023, a $118 million tax charge increased the valuation allowance on Swiss deferred tax assets, leading to a higher effective tax rate.',
    'What accounted for the significant tax rate increase in fiscal year 2023?',
    'What percentage of the box office revenue in the U.S./Canada was generated by the three largest exhibitors in 2023?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

Size: 6,300 training samples
Columns: positive and anchor
Approximate statistics based on the first 1000 samples:
positive anchor
type string string
details
min: 2 tokens
mean: 46.25 tokens
max: 512 tokens

min: 2 tokens
mean: 20.35 tokens
max: 51 tokens

	positive	anchor
type	string	string
details	min: 2 tokens mean: 46.25 tokens max: 512 tokens	min: 2 tokens mean: 20.35 tokens max: 51 tokens

Samples:

positive	anchor
`For the year ended December 31, 2023, net cash used in financing activities included $1.8 billion for dividends to GM, which are eliminated within the consolidated statements of cash flows.`	`What amount of dividends to GM were included in the net cash used in financing activities for GM Financial for the year ended December 31, 2023?`
`Assets and liabilities of these foreign entities are translated at exchange rates in effect as of the balance sheet date.`	`At what values are assets and liabilities of foreign entities translated in financial statements?`
`The 21st Century Cures Act broadened patient access to certain enhanced benefits offered by Medicare Advantage plans, increasing the percentage of patients on these plans.`	`How did the 21st Century Cures Act affect patient access to Medicare Advantage plans?`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Framework Versions

Python: 3.10.14
Sentence Transformers: 3.0.1
Transformers: 4.41.2
PyTorch: 2.1.2+cu121
Accelerate: 0.33.0
Datasets: 2.19.1
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}