--- base_model: Snowflake/snowflake-arctic-embed-l library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 - dot_accuracy@1 - dot_accuracy@3 - dot_accuracy@5 - dot_accuracy@10 - dot_precision@1 - dot_precision@3 - dot_precision@5 - dot_precision@10 - dot_recall@1 - dot_recall@3 - dot_recall@5 - dot_recall@10 - dot_ndcg@10 - dot_mrr@10 - dot_map@100 pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:400 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss widget: - source_sentence: What does it mean for a decision to not be considered arbitrary and capricious according to the provided context? sentences: - "Aravind Srinivas, Founding Story and Journey of Perplexity, YOUTUBE, at 17:57\ \ (Jan. 18, 2024), \nhttps://www.youtube.com/watch?v=ygRVDIwheB4. \n17 See, e.g.,\ \ Avoiding Plagiarism Guide, APA Style 7th Edition (last visited Aug. 30, 2024),\ \ \nhttps://apastyle.apa.org/instructional-aids/avoiding-plagiarism.pdf. \n18\ \ See What is Perplexity?, supra note 1 (promoting Perplexity’s “Reliable sources”\ \ with an \nexplanation that “[e]very answer is backed by citations from trusted\ \ news outlets, academic papers, \nand established blogs”). \n19 Madhumita Murgia\ \ & Cristina Criddle, Perplexity’s popularity surges as AI search start-up takes\ \ \non Google, THE FINANCIAL TIMES (Aug. 9, 2024), https://www.ft.com/content/87af3340-2611-\n\ 4650-9ae3-036927e9f65c." - "30 \n \nServ. Comm'n, 43 Mass. App. Ct. 300, 303 (1997). A decision is not arbitrary\ \ and capricious if \n\"reasonable minds could differ\" on the proper outcome.\ \ See Kinchla v. Board of Appeals of \nFalmouth, 11 Mass. App. Ct. 927, 927 (1981).\ \ \nIn determining the appropriate definition of general words used in a statute,\ \ the courts may \nlook to sources outside the statute such as \"their use in\ \ other legal contexts: and dictionary \ndefinitions.\" See Commonwealth v. Correia,\ \ 17 Mass.App.Ct. 233, 235 (1983) “Arbitrary” is \ndefined as subject to individual\ \ will or judgment without restriction; contingent solely upon one's \ndiscretion…\ \ having unlimited power; uncontrolled or unrestricted by law; despotic; tyrannical;" - "purpose of providing a substitute product. \nCase 1:24-cv-07984 Document\ \ 1 Filed 10/21/24 Page 3 of 42" - source_sentence: What percentage of applicants were admitted to Stanford last year? sentences: - "to which RNH is currently applying are extremely competitive and the admissions\ \ process for \nadmission into such schools is rigorous. These schools command\ \ an extensive applicant pool of \nhigh academic achievers with high test scores,\ \ grade point averages, including grades of A’s and \nB’s only. Stanford is one\ \ of the most competitive schools in the country. Last year, 4% of the \napplicant\ \ pool were admitted. Thousands of extremely well qualified, who elsewhere would\ \ be \nhighly admissible, were denied. It is essential that any applicant have\ \ the most competitive \ntranscript possible. A C+ is a red flag that will be\ \ noticed far more quickly and glaringly than the \nCase 1:24-cv-12437-WGY Document\ \ 8 Filed 10/08/24 Page 6 of 42" - "18 \n \nupon in affirming the decision through an appeal to exclude RNH and his\ \ classmate from the NHS. \nId. at ¶145. At that time, Defendant Swanson and\ \ other Defendants knew or should have known \nthat the District inducted at least\ \ seven students into NHS, who had academic infractions on their \nrecord, one\ \ of which was because of the prior use of AI. Id. at ¶146. \nThe “committee”\ \ that adjudicated selection for NHS this year did not include teachers who \n\ know and are familiar with RNH and his classmate. Id. at ¶147. This is due to\ \ the then escalating \ncontract conflict with the Hingham Educators Association\ \ (“HEA”) where HEA engaged in an" - "42 \n \nCERTIFICATE OF SERVICE \n \nI, Peter S. Farrell, hereby certify that\ \ I served a copy of the foregoing on all counsel of \nrecord pursuant to Local\ \ Rule 5.4(c) by causing a copy of the same to be electronically filed and \n\ served through the CM/ECF filing system to: \n \nGareth W. Notis, Esquire \nMorrison\ \ Mahoney LLP \n250 Summer Street \nBoston, MA 02210 \ngnotis@morrisonmahoney.com\ \ \n \n \n \n \n \n \n \n \n______________________________ \n \n \n \n \n \n \n\ Peter S. Farrell \n \nCase 1:24-cv-12437-WGY Document 8 Filed 10/08/24 Page\ \ 42 of 42" - source_sentence: What is the case number for the document filed on 10/08/24? sentences: - Case 1:24-cv-07984 Document 1 Filed 10/21/24 Page 19 of 42 - Case 1:24-cv-12437-WGY Document 8 Filed 10/08/24 Page 33 of 42 - "11 See, e.g., Elizabeth Lopatto, Perplexity’s Grand Theft AI, THE VERGE (June\ \ 27, 2024), \nhttps://www.theverge.com/2024/6/27/24187405/perplexity-ai-twitter-lie-plagiarism\ \ \n(describing \nPerplexity as a “rent-seeking middleman on high-quality sources”\ \ that “starve[s] the primary \nsource of ad revenue”); Dhruv Mehrotra & Tim Marchman,\ \ Perplexity Is a Bullsh*t Machine, \nWIRED \n(June \n19, \n2024), \nhttps://www.wired.com/story/perplexity-is-a-bullshit-machine\ \ \n(discussing Perplexity’s reliance on recent news articles for its content\ \ as well as its tendency to \nfalsely attribute information) (asterisk added);\ \ Casey Newton, How to Stop Perplexity and save \nthe web from bad AI, PLATFORMER\ \ (June 20, 2024), https://www.platformer.news/how-to-stop-" - source_sentence: How does Perplexity gather and compile information from authoritative sources? sentences: - "utilize have been trained. To employ a RAG system, AI applications typically\ \ utilize indexed \ndatabases that house all the content from which the AI application\ \ will retrieve specific information \nto generate outputs for its users. The\ \ larger the index, the more “answers” the AI application can \nprovide. \n51.\ \ \nIn Perplexity’s words, it “scours the internet, gathering information from\ \ \nauthoritative sources like articles, websites, and journals.”6 It then, “compiles\ \ the most relevant \ninsights into a coherent, easy-to-understand answer” automatically\ \ generated from those original \nsources.7 \n52. \nThe assembling of authoritative\ \ sources for a RAG index is a distinct process from" - "9 \n26. \nPerplexity processes subscription purchases from customers in this\ \ State and \nDistrict, transmits Plaintiffs’ copyrighted content to users in\ \ this State and District, and has a \nsignificant number of customers in this\ \ State and District. \n27. \nAs a direct and proximate result of Perplexity’s\ \ unauthorized use and/or \ndissemination of Plaintiffs’ copyrighted works and\ \ trademarks in New York and elsewhere, \nPlaintiffs have lost and will continue\ \ to lose revenue and profits from the market for content \nlicensing, subscribers,\ \ visitors, and users. \nFACTUAL ALLEGATIONS \nI. \nPlaintiffs’ Robust Businesses\ \ and Copyrighted Works \n28. \nDow Jones began in 1882 as a niche news agency\ \ in a Wall Street basement," - "1 \nUNITED STATES DISTRICT COURT \nSOUTHERN DISTRICT OF NEW YORK \n \nDOW JONES\ \ & COMPANY, INC. \nand NYP HOLDINGS, INC., \n \nPlaintiffs, \n \nv. \n \nPERPLEXITY\ \ AI, INC., \n \nDefendant. \n \n \n \nCivil Action No. 24-cv-7984 \n \n \nCOMPLAINT\ \ \n \nJURY TRIAL DEMANDED \n \nPlaintiffs Dow Jones & Company, Inc. (“Dow Jones”)\ \ and NYP Holdings, Inc. (“NYP \nHoldings”) (collectively, “Plaintiffs”), by and\ \ through their attorneys, Torridon Law PLLC, for \ntheir Complaint, hereby allege\ \ against Defendant Perplexity AI, Inc. (“Perplexity” or \n“Defendant”), as follows:\ \ \nNATURE OF THE ACTION \n1. \nPerplexity is a generative artificial intelligence\ \ company that claims to provide its" - source_sentence: What recent partnership did News Corp enter into regarding licensing content for OpenAI's applications? sentences: - "integrity infractions. Plain and simple. It should not take the Plaintiffs\ \ engaging counsel, \ndemanding information and forcing Hingham to investigate\ \ this matter to reveal that selection for \nNHS was a manipulated sham conducted\ \ by the Defendants, who at all times relevant were state \nactors. \nC. The Student\ \ Will Suffer Irreparable Harm If The Injunction is Not Granted \nIn order for\ \ the Plaintiffs to obtain injunctive relief, they must show that they are \"\ likely to \nsuffer irreparable injury before a decision is rendered on the merits.\"\ \ See Philips Elecs. N. Am. \nCorp. v. Halperin, 2000 Mass. Super LEXIS 574 citing\ \ Sierra Club v. Larson, 769 F. Supp. 420," - "licensing initiatives abound.”3 For example, News Corp recently partnered with\ \ OpenAI to license \nits content for certain uses in OpenAI’s applications. OpenAI\ \ users will have the benefit of \naccessing Plaintiffs’ content, whether quoted\ \ or summarized by OpenAI. This cooperative \nrelationship will allow OpenAI and\ \ Plaintiffs to experiment with new product experiences and \nrevenue models.\ \ \n15. \nGenerative AI technology can be developed in two ways. It can be developed\ \ \nlegally by recognizing the legitimate rights of copyright holders and by including\ \ in the AI business \nmodel the legitimate costs and benefits of licensing the\ \ copyrighted material, or it can be developed" - "ban or prohibition on the use of AI by students. The Defendants were not trained\ \ on any policies \nor procedures for use of AI alone, never mind what they were\ \ “able to do” to students who used \nit. The entire purpose behind having\ \ such policies and procedures in place is to ensure notice, \nequity, fairness\ \ and to be sure: a level playing field for all. Making matters worse, there\ \ exists \nno adequate procedures and policies for the induction of an applicant\ \ into NHS when compared to \nother members who are inducted despite the same\ \ or similar infractions. This is a denial of student \nrights of the highest\ \ order. \n \nIn the case here, RNH was disciplined on an ad hoc and on-going\ \ basis over more than six" model-index: - name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l results: - task: type: information-retrieval name: Information Retrieval dataset: name: Unknown type: unknown metrics: - type: cosine_accuracy@1 value: 0.6875 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.8541666666666666 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.9583333333333334 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9791666666666666 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.6875 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.28472222222222215 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.19166666666666665 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09791666666666665 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.6875 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.8541666666666666 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.9583333333333334 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9791666666666666 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8280840444145441 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7793650793650793 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7812590187590187 name: Cosine Map@100 - type: dot_accuracy@1 value: 0.6875 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.8541666666666666 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.9583333333333334 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.9791666666666666 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.6875 name: Dot Precision@1 - type: dot_precision@3 value: 0.28472222222222215 name: Dot Precision@3 - type: dot_precision@5 value: 0.19166666666666665 name: Dot Precision@5 - type: dot_precision@10 value: 0.09791666666666665 name: Dot Precision@10 - type: dot_recall@1 value: 0.6875 name: Dot Recall@1 - type: dot_recall@3 value: 0.8541666666666666 name: Dot Recall@3 - type: dot_recall@5 value: 0.9583333333333334 name: Dot Recall@5 - type: dot_recall@10 value: 0.9791666666666666 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.8280840444145441 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.7793650793650793 name: Dot Mrr@10 - type: dot_map@100 value: 0.7812590187590187 name: Dot Map@100 --- # SentenceTransformer based on Snowflake/snowflake-arctic-embed-l This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 1024 tokens - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("llm-wizard/legal-ft-arctic-l") # Run inference sentences = [ "What recent partnership did News Corp enter into regarding licensing content for OpenAI's applications?", 'licensing initiatives abound.”3 For example, News Corp recently partnered with OpenAI to license \nits content for certain uses in OpenAI’s applications. OpenAI users will have the benefit of \naccessing Plaintiffs’ content, whether quoted or summarized by OpenAI. This cooperative \nrelationship will allow OpenAI and Plaintiffs to experiment with new product experiences and \nrevenue models. \n15. \nGenerative AI technology can be developed in two ways. It can be developed \nlegally by recognizing the legitimate rights of copyright holders and by including in the AI business \nmodel the legitimate costs and benefits of licensing the copyrighted material, or it can be developed', 'integrity infractions. Plain and simple. It should not take the Plaintiffs engaging counsel, \ndemanding information and forcing Hingham to investigate this matter to reveal that selection for \nNHS was a manipulated sham conducted by the Defendants, who at all times relevant were state \nactors. \nC. The Student Will Suffer Irreparable Harm If The Injunction is Not Granted \nIn order for the Plaintiffs to obtain injunctive relief, they must show that they are "likely to \nsuffer irreparable injury before a decision is rendered on the merits." See Philips Elecs. N. Am. \nCorp. v. Halperin, 2000 Mass. Super LEXIS 574 citing Sierra Club v. Larson, 769 F. Supp. 420,', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 1024] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.6875 | | cosine_accuracy@3 | 0.8542 | | cosine_accuracy@5 | 0.9583 | | cosine_accuracy@10 | 0.9792 | | cosine_precision@1 | 0.6875 | | cosine_precision@3 | 0.2847 | | cosine_precision@5 | 0.1917 | | cosine_precision@10 | 0.0979 | | cosine_recall@1 | 0.6875 | | cosine_recall@3 | 0.8542 | | cosine_recall@5 | 0.9583 | | cosine_recall@10 | 0.9792 | | cosine_ndcg@10 | 0.8281 | | cosine_mrr@10 | 0.7794 | | **cosine_map@100** | **0.7813** | | dot_accuracy@1 | 0.6875 | | dot_accuracy@3 | 0.8542 | | dot_accuracy@5 | 0.9583 | | dot_accuracy@10 | 0.9792 | | dot_precision@1 | 0.6875 | | dot_precision@3 | 0.2847 | | dot_precision@5 | 0.1917 | | dot_precision@10 | 0.0979 | | dot_recall@1 | 0.6875 | | dot_recall@3 | 0.8542 | | dot_recall@5 | 0.9583 | | dot_recall@10 | 0.9792 | | dot_ndcg@10 | 0.8281 | | dot_mrr@10 | 0.7794 | | dot_map@100 | 0.7813 | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 400 training samples * Columns: sentence_0 and sentence_1 * Approximate statistics based on the first 400 samples: | | sentence_0 | sentence_1 | |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | sentence_0 | sentence_1 | |:---------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | How does Perplexity's business model differ from that of traditional search engines? | 11.
Perplexity’s business is fundamentally distinct from that of traditional search
engines that also copy a vast amount of content into their indices but do so merely to provide links
to the originating sites. In its traditional form, a search engine is a tool for discovery, pointing
searchers to websites such as the pages of The Wall Street Journal or the New York Post, where the
users can click to find the information and answers they seek. Those clicks in turn provide revenue
for content producers. In part because traditional search engines that simply provide hyperlinks
promote merely the discovery of copyrighted content, and not its substitution (and commercial | | What role do clicks on traditional search engines play in the revenue generation for content producers? | 11.
Perplexity’s business is fundamentally distinct from that of traditional search
engines that also copy a vast amount of content into their indices but do so merely to provide links
to the originating sites. In its traditional form, a search engine is a tool for discovery, pointing
searchers to websites such as the pages of The Wall Street Journal or the New York Post, where the
users can click to find the information and answers they seek. Those clicks in turn provide revenue
for content producers. In part because traditional search engines that simply provide hyperlinks
promote merely the discovery of copyrighted content, and not its substitution (and commercial | | Who were the founders of Dow Jones? | founded by reporters Charles Dow, Edward Jones, and Charles Bergstresser. Publishing the first
edition of The Wall Street Journal in July 1889, Dow Jones has now expanded into a worldwide
news powerhouse. It creates and distributes some of the most widely recognized and reputable
publications in the news industry, including, in addition to The Wall Street Journal, Dow Jones
Newswires, MarketWatch, Financial News, and Barron’s.
29.
Dow Jones is a trusted source of accurate, original news stories, data and analytics,
and financial and business insight for millions of customers across the country and around the
world.
30.
A recipient of 39 Pulitzer Prizes, the award-winning newsroom at The Wall Street | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 768, 512, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 10 - `per_device_eval_batch_size`: 10 - `num_train_epochs`: 10 - `multi_dataset_batch_sampler`: round_robin #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 10 - `per_device_eval_batch_size`: 10 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 5e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1 - `num_train_epochs`: 10 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.0 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `eval_use_gather_object`: False - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: round_robin
### Training Logs | Epoch | Step | cosine_map@100 | |:-----:|:----:|:--------------:| | 1.0 | 40 | 0.7519 | | 1.25 | 50 | 0.8072 | | 2.0 | 80 | 0.7892 | | 2.5 | 100 | 0.7949 | | 3.0 | 120 | 0.7850 | | 3.75 | 150 | 0.7537 | | 4.0 | 160 | 0.7905 | | 5.0 | 200 | 0.7650 | | 6.0 | 240 | 0.7860 | | 6.25 | 250 | 0.7806 | | 7.0 | 280 | 0.7819 | | 7.5 | 300 | 0.7820 | | 8.0 | 320 | 0.7820 | | 8.75 | 350 | 0.7821 | | 9.0 | 360 | 0.7823 | | 10.0 | 400 | 0.7813 | ### Framework Versions - Python: 3.10.12 - Sentence Transformers: 3.2.1 - Transformers: 4.44.2 - PyTorch: 2.4.1+cu121 - Accelerate: 0.34.2 - Datasets: 3.0.2 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```