--- base_model: BAAI/bge-base-en-v1.5 datasets: [] language: - en library_name: sentence-transformers license: apache-2.0 pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:6300 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss widget: - source_sentence: Consumer Products segment decreased 10% to $3,572.5 million. sentences: - What was the impact of the Federal Reserve’s policy changes on Schwab money market funds in 2022? - What was the total revenue of Hasbro's Consumer Products segment in 2022? - How much did the company's currently payable U.S. taxes amount to in 2023? - source_sentence: PricewaterhouseCoopers LLP is mentioned as the Firm’s independent registered public accounting firm (PCAOB ID 238) in the audit of the Consolidated Financial Statements. sentences: - Where in the document can the Consolidated Financial Statements be found as mentioned in a 2024 report? - What type of firm is PricewaterhouseCoopers LLP as mentioned in the context of auditing? - Which note in the report provides details about legal proceedings? - source_sentence: If, in the future, foreign exchange or capital control restrictions were to be imposed and become applicable to us, such restrictions could potentially reduce the amounts that we would be able to receive from our Macao, Hong Kong and mainland China subsidiaries. sentences: - What are the potential consequences for the parent company if foreign exchange or capital control restrictions were imposed in the future? - What is described under Item 8 in the context of a financial document? - What types of investments are primarily included in the Goldman Sachs' investments in funds at NAV as of December 2023? - source_sentence: Determining income tax provisions involves forecasting future financial results, planning potential tax strategies, and evaluating the probability of sustaining tax positions against audits. sentences: - What type of company is Johnson & Johnson described as? - What determines the fair value of available-for-sale short-term investments? - What factors influence the determination of income tax provisions and related tax balances? - source_sentence: During the fiscal year ended March 31, 2023, a $118 million tax charge increased the valuation allowance on Swiss deferred tax assets, leading to a higher effective tax rate. sentences: - What accounted for the significant tax rate increase in fiscal year 2023? - What percentage of the box office revenue in the U.S./Canada was generated by the three largest exhibitors in 2023? - What percentage of eBay's 2023 net revenues were attributed to international markets? --- # BGE base Financial Matryoshka This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 tokens - **Similarity Function:** Cosine Similarity - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("ValentinaKim/bge-base-financial-matryoshka4") # Run inference sentences = [ 'During the fiscal year ended March 31, 2023, a $118 million tax charge increased the valuation allowance on Swiss deferred tax assets, leading to a higher effective tax rate.', 'What accounted for the significant tax rate increase in fiscal year 2023?', 'What percentage of the box office revenue in the U.S./Canada was generated by the three largest exhibitors in 2023?', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 6,300 training samples * Columns: positive and anchor * Approximate statistics based on the first 1000 samples: | | positive | anchor | |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | positive | anchor | |:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------| | For the year ended December 31, 2023, net cash used in financing activities included $1.8 billion for dividends to GM, which are eliminated within the consolidated statements of cash flows. | What amount of dividends to GM were included in the net cash used in financing activities for GM Financial for the year ended December 31, 2023? | | Assets and liabilities of these foreign entities are translated at exchange rates in effect as of the balance sheet date. | At what values are assets and liabilities of foreign entities translated in financial statements? | | The 21st Century Cures Act broadened patient access to certain enhanced benefits offered by Medicare Advantage plans, increasing the percentage of patients on these plans. | How did the 21st Century Cures Act affect patient access to Medicare Advantage plans? | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 768, 512, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Framework Versions - Python: 3.10.14 - Sentence Transformers: 3.0.1 - Transformers: 4.41.2 - PyTorch: 2.1.2+cu121 - Accelerate: 0.33.0 - Datasets: 2.19.1 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```