batch32-100 / README.md

Add new SentenceTransformer model.

4768a66 verified 2 months ago

26.7 kB

	---
	base_model: intfloat/multilingual-e5-small
	datasets: []
	language: []
	library_name: sentence-transformers
	metrics:
	- cosine_accuracy@1
	- cosine_accuracy@3
	- cosine_accuracy@5
	- cosine_accuracy@10
	- cosine_precision@1
	- cosine_precision@3
	- cosine_precision@5
	- cosine_precision@10
	- cosine_recall@1
	- cosine_recall@3
	- cosine_recall@5
	- cosine_recall@10
	- cosine_ndcg@10
	- cosine_mrr@10
	- cosine_map@100
	pipeline_tag: sentence-similarity
	tags:
	- sentence-transformers
	- sentence-similarity
	- feature-extraction
	- generated_from_trainer
	- dataset_size:16
	- loss:MultipleNegativesRankingLoss
	widget:
	- source_sentence: Please contact each construction office of the Construction Bureau.
	The police will respond in cooperation with the police and other authorities.
	For the telephone number of each construction office, please refer to the link
	"Area Management Offices and Construction Offices".
	sentences:
	- I have an abandoned vehicle on the street, what should I do?
	- Do I need special permission to place a giant ad?
	- What is katagatashi/tai?
	- source_sentence: Currently, there are four Seseragi-no-Sato (Seseragi-no-Sato) oases
	in the city filled with flowers and greenery, as an oasis of relaxation and luster.
	How about touring the "Seseragi-no-Sato" while admiring the seasonal flowers?
	Please take a stroll. For more information, please refer to the link "Flower Sewage
	Treatment Plant and Seseragi no Sato (Sewage Treatment Plant and Seseragi no Sato)".
	For more information, please refer to the link "Sewage Treatment Plants and Seseragi
	no Sato (Flower Sewage Treatment Plants and Seseragi no Sato)".
	sentences:
	- I want to install a sign on the road. Do I need any permits?
	- Who can I talk to about housing?
	- I would like to know more about Seseragi no Sato.
	- source_sentence: 'The Osaka Municipal Housing Information Center provides comprehensive
	information on housing, and consists of the Housing Information Plaza, which provides
	various consultations and information on housing, and the Osaka Kurashi-no Konjikan,
	a museum of housing that exhibits the culture and history of housing and people''s
	lives. Location and Access] Location: 6-4-20 Tenjinbashi, Kita-ku, Osaka Access:
	- Direct connection from Exit 3 of Tenjinbashisuji Rokuchome Station on the Osaka
	Metro Tanimachi Line, Sakaisuji Line, and Hankyu Railway - Approximately 650 m
	north of Tenma Station on the JR Loop Line - Approximately 2 km by cab from Midosuji
	South Exit of JR Osaka Station via Miyakojima-dori, 7 minutes by car By car: Approx.
	500 m from Nagara Exit on the Moriguchi Line of Hanshin Expressway via Miyakojima-dori
	Street. ◆Housing Information Plaza Hours: Weekdays and Saturdays: 9:00-19:00,
	Sundays and National Holidays: 10:00-17:00, Closed: Tuesdays (closed the following
	day if Tuesday is a national vacation), the day after national holidays (except
	Sundays and Mondays), year-end and New Year holidays (12/29 - 1/3) *Special holidays
	may occur in addition to the above. ◆Housing Museum "Osaka Kurashi no Konjakukan"
	Hours: 10:00 - 17:00 (admission until 16:30) Closed: Tuesdays, Year-end and New
	Year holidays (12/29 - 1/3) *The museum may be open or closed on a temporary basis
	in addition to the above. In addition to the above, the museum may be open or
	closed temporarily. 6208-9224 Fax: 06-6202-7064'
	sentences:
	- Please tell me about the Osaka Municipal Housing Information Center.
	- Where is advertising prohibited?
	- How much is the admission fee to Osaka Kurashi-no-Museum?
	- source_sentence: 'A pamphlet and leaflet, "Sewerage in Osaka City," which introduces
	the sewerage system of Osaka City, including its structure and roles, are distributed
	at City Hall and other locations. They are also available on the city website.
	You can also tour the following sewerage facilities. All tours are free of charge.
	Taikoh Sewer: You can visit the Taikoh Sewer, a designated cultural asset of Osaka
	City. Those who wish to tour the underground facilities must apply in advance.
	Maishima Sludge Center Sludge Treatment Facility】Persons wishing to tour the sludge
	treatment facility are required to apply in advance. Sewage Treatment Plants】Persons
	wishing to tour the facilities should contact the respective sewage treatment
	plant in advance. (Tours may not be available due to construction work at sewage
	treatment plants.) For details, please refer to the following links: "Leaflet
	"Sewerage in Osaka City" (digest version)," "Pamphlet "Sewerage in Osaka City,"
	"Taikoh Sewage Treatment Plant," "Maishima Sludge Center (sewage sludge treatment
	plant)," "Osaka City Visual Sewage Plan," "Osaka Eco Kids: Learn Sewage! for more
	information.'
	sentences:
	- I want to know how the sewage system works.
	- How much is the rent for Osaka City's excellent rental housing for the elderly?
	- Please tell me about K.K. General Rental Housing and K.K. Sumai Ringu.
	- source_sentence: For posters, billboards, etc. on roads that are in violation of
	the ordinance, we systematically provide corrective guidance and remove them,
	as well as conduct road patrols as needed. In addition, for minor violations such
	as posters and billboards, we also remove them through the activities of contractors
	and citizen volunteers called "Katayaki-Tai". For details, please refer to the
	link "Recruitment of "KATADACHI-TAI" (a system for removing simple advertisements
	on the street)". For more information, please refer to the link "Simple Roadside
	Advertisement Removal Activities" for details.
	sentences:
	- I am thinking of buying a house in Osaka City, is there any assistance available?
	(Newlyweds and families raising children)
	- What is the maximum size of a billboard or advertisement?
	- What measures are in place to deal with objectionable signs and posters?
	model-index:
	- name: SentenceTransformer based on intfloat/multilingual-e5-small
	results:
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: intfloat/multilingual e5 small
	type: intfloat/multilingual-e5-small
	metrics:
	- type: cosine_accuracy@1
	value: 0.75
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 1.0
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 1.0
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 1.0
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.75
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.3333333333333333
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.2
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.1
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.75
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 1.0
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 1.0
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 1.0
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.9077324383928644
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.875
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.875
	name: Cosine Map@100
	---

	# SentenceTransformer based on intfloat/multilingual-e5-small

	This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

	## Model Details

	### Model Description
	- Model Type: Sentence Transformer
	- Base model: [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small) <!-- at revision fd1525a9fd15316a2d503bf26ab031a61d056e98 -->
	- Maximum Sequence Length: 512 tokens
	- Output Dimensionality: 384 tokens
	- Similarity Function: Cosine Similarity
	<!-- - Training Dataset: Unknown -->
	<!-- - Language: Unknown -->
	<!-- - License: Unknown -->

	### Model Sources

	- Documentation: [Sentence Transformers Documentation](https://sbert.net)
	- Repository: [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
	- Hugging Face: [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

	### Full Model Architecture

	```
	SentenceTransformer(
	(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
	(1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
	(2): Normalize()
	)
	```

	## Usage

	### Direct Usage (Sentence Transformers)

	First install the Sentence Transformers library:

	```bash
	pip install -U sentence-transformers
	```

	Then you can load this model and run inference.
	```python
	from sentence_transformers import SentenceTransformer

	# Download from the 🤗 Hub
	model = SentenceTransformer("Yohhei/batch32-100")
	# Run inference
	sentences = [
	'For posters, billboards, etc. on roads that are in violation of the ordinance, we systematically provide corrective guidance and remove them, as well as conduct road patrols as needed. In addition, for minor violations such as posters and billboards, we also remove them through the activities of contractors and citizen volunteers called "Katayaki-Tai". For details, please refer to the link "Recruitment of "KATADACHI-TAI" (a system for removing simple advertisements on the street)". For more information, please refer to the link "Simple Roadside Advertisement Removal Activities" for details.',
	'What measures are in place to deal with objectionable signs and posters?',
	'I am thinking of buying a house in Osaka City, is there any assistance available? (Newlyweds and families raising children)',
	]
	embeddings = model.encode(sentences)
	print(embeddings.shape)
	# [3, 384]

	# Get the similarity scores for the embeddings
	similarities = model.similarity(embeddings, embeddings)
	print(similarities.shape)
	# [3, 3]
	```

	<!--
	### Direct Usage (Transformers)

	<details><summary>Click to see the direct usage in Transformers</summary>

	</details>
	-->

	<!--
	### Downstream Usage (Sentence Transformers)

	You can finetune this model on your own dataset.

	<details><summary>Click to expand</summary>

	</details>
	-->

	<!--
	### Out-of-Scope Use

	List how the model may foreseeably be misused and address what users ought not to do with the model.
	-->

	## Evaluation

	### Metrics

	#### Information Retrieval
	* Dataset: `intfloat/multilingual-e5-small`
	* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

	\| Metric \| Value \|
	\|:--------------------\|:----------\|
	\| cosine_accuracy@1 \| 0.75 \|
	\| cosine_accuracy@3 \| 1.0 \|
	\| cosine_accuracy@5 \| 1.0 \|
	\| cosine_accuracy@10 \| 1.0 \|
	\| cosine_precision@1 \| 0.75 \|
	\| cosine_precision@3 \| 0.3333 \|
	\| cosine_precision@5 \| 0.2 \|
	\| cosine_precision@10 \| 0.1 \|
	\| cosine_recall@1 \| 0.75 \|
	\| cosine_recall@3 \| 1.0 \|
	\| cosine_recall@5 \| 1.0 \|
	\| cosine_recall@10 \| 1.0 \|
	\| cosine_ndcg@10 \| 0.9077 \|
	\| cosine_mrr@10 \| 0.875 \|
	\| cosine_map@100 \| 0.875 \|

	<!--
	## Bias, Risks and Limitations

	What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.
	-->

	<!--
	### Recommendations

	What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.
	-->

	## Training Details

	### Training Dataset

	#### Unnamed Dataset


	* Size: 16 training samples
	* Columns: <code>positive</code> and <code>anchor</code>
	* Approximate statistics based on the first 1000 samples:
	\| \| positive \| anchor \|
	\|:--------\|:-------------------------------------------------------------------------------------\|:----------------------------------------------------------------------------------\|
	\| type \| string \| string \|
	\| details \| <ul><li>min: 51 tokens</li><li>mean: 226.69 tokens</li><li>max: 419 tokens</li></ul> \| <ul><li>min: 8 tokens</li><li>mean: 16.81 tokens</li><li>max: 31 tokens</li></ul> \|
	* Samples:
	\| positive \| anchor \|
	\|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\|:------------------------------------------------------------------------------------\|
	\| <code>The following is a summary of K.H.I. General Rental Housing and K.H.I. Sumai Ringu. Outline】 ● KLH General Rental Housing and KLH Sumai Ringu are rental housing units for middle class households where the annual income of the applicant household must be within a certain range. The public corporation's Sumai Ringu housing is a unit to which the government's specified excellent rental housing system, etc., is applied, and the government and Osaka City subsidize a portion of the rent for a certain period of time, depending on the income of the household moving in. Applications for vacant units are accepted on an as-needed basis. Please refer to the "List of Apartment Complexes" at the link below. Inquiries: ◆Osaka City Housing Corporation, Housing Management Department, Management Division, Recruitment Section Telephone: 06-6882-9000* Weekdays: 9:00 - 19:00 (Tuesdays and the day after national holidays (weekdays): 9:00 - 17:30) Saturdays: 9:00 - 19:00 Sundays and holidays: 10:00 - 17:00, except during the year-end and New Year holidays (December 29 - January 3).</code> \| <code>Please tell me about K.K. General Rental Housing and K.K. Sumai Ringu.</code> \|
	\| <code>There are permit criteria for each property for wall boards, towers (rooftop/ground), and boards (rooftop/ground), and permit criteria vary by location. Please refer to the "Permit Criteria" in the "Outdoor Advertisement Bookmark. (Downloadable from the website) △Link to "https://www.city.osaka.lg.jp/kensetsu/page/0000372127.html屋外広告物の許可について (Outdoor Advertisement Bookmark, Outdoor Advertisement Ordinance, etc.) [Inquiries] ◆Construction Bureau, Administration Division Phone: 06-6615-6687 Fax: 06-6615-6576</code> \| <code>What is the maximum size of a billboard or advertisement?</code> \|
	\| <code>Areas or properties where advertising materials may not be displayed are as follows In addition to the above, the following areas or properties are prohibited from displaying advertising materials: - Areas along the Hanshin Expressway up to 50 m on both sides and 15 m above the road surface level - Areas within the grounds of ancient tombs and cemeteries - Bridges, roadside trees, traffic signals, pedestrian railings, utility poles, mailboxes, transmission towers, statues, monuments, etc. - The Okawa Wind Area from Genpachi Bridge to Tenmabashi Bridge. In addition to the above, the display of posters, billboards, etc., advertising flags, and standing signs, etc., is prohibited on the following roads and in areas or locations facing these roads. Midosuji (from Osaka Station to Namba Station) ●Sakaisuji (from Naniwabashi to Nipponbashi) ●Tosabori Dori (from Higobashi to Yoshiyabashi) ●Uemachi-suji (from Otemae 1-chome, Chuo-ku to Hoenzaka 1-chome, Chuo-ku) ●Nagahori Dori (from Minami-Senba 1-chome, Chuo-ku to Minami Senba 1-chome, Chuo-ku) Dotonbori River promenade (from east side of Sumiyoshi Bridge to west side of Nihonbashi Bridge)-Please refer to the link. https://www.city.osaka.lg.jp/kensetsu/page/ 0000372127.htmlAbout Permission for Outdoor Advertisements (Outdoor Advertisement Booklet, Outdoor Advertisement Ordinance, etc.) [Inquiries: ◆Construction Bureau, Administration Division Tel: 06-6615-6687 Fax: 06-6615-6576</code> \| <code>Where is advertising prohibited?</code> \|
	* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
	```json
	{
	"scale": 20.0,
	"similarity_fct": "cos_sim"
	}
	```

	### Training Hyperparameters
	#### Non-Default Hyperparameters

	- `eval_strategy`: steps
	- `per_device_train_batch_size`: 32
	- `per_device_eval_batch_size`: 16
	- `learning_rate`: 2e-05
	- `num_train_epochs`: 1
	- `warmup_ratio`: 0.1
	- `fp16`: True
	- `batch_sampler`: no_duplicates

	#### All Hyperparameters
	<details><summary>Click to expand</summary>

	- `overwrite_output_dir`: False
	- `do_predict`: False
	- `eval_strategy`: steps
	- `prediction_loss_only`: True
	- `per_device_train_batch_size`: 32
	- `per_device_eval_batch_size`: 16
	- `per_gpu_train_batch_size`: None
	- `per_gpu_eval_batch_size`: None
	- `gradient_accumulation_steps`: 1
	- `eval_accumulation_steps`: None
	- `torch_empty_cache_steps`: None
	- `learning_rate`: 2e-05
	- `weight_decay`: 0.0
	- `adam_beta1`: 0.9
	- `adam_beta2`: 0.999
	- `adam_epsilon`: 1e-08
	- `max_grad_norm`: 1.0
	- `num_train_epochs`: 1
	- `max_steps`: -1
	- `lr_scheduler_type`: linear
	- `lr_scheduler_kwargs`: {}
	- `warmup_ratio`: 0.1
	- `warmup_steps`: 0
	- `log_level`: passive
	- `log_level_replica`: warning
	- `log_on_each_node`: True
	- `logging_nan_inf_filter`: True
	- `save_safetensors`: True
	- `save_on_each_node`: False
	- `save_only_model`: False
	- `restore_callback_states_from_checkpoint`: False
	- `no_cuda`: False
	- `use_cpu`: False
	- `use_mps_device`: False
	- `seed`: 42
	- `data_seed`: None
	- `jit_mode_eval`: False
	- `use_ipex`: False
	- `bf16`: False
	- `fp16`: True
	- `fp16_opt_level`: O1
	- `half_precision_backend`: auto
	- `bf16_full_eval`: False
	- `fp16_full_eval`: False
	- `tf32`: None
	- `local_rank`: 0
	- `ddp_backend`: None
	- `tpu_num_cores`: None
	- `tpu_metrics_debug`: False
	- `debug`: []
	- `dataloader_drop_last`: False
	- `dataloader_num_workers`: 0
	- `dataloader_prefetch_factor`: None
	- `past_index`: -1
	- `disable_tqdm`: False
	- `remove_unused_columns`: True
	- `label_names`: None
	- `load_best_model_at_end`: False
	- `ignore_data_skip`: False
	- `fsdp`: []
	- `fsdp_min_num_params`: 0
	- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
	- `fsdp_transformer_layer_cls_to_wrap`: None
	- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
	- `deepspeed`: None
	- `label_smoothing_factor`: 0.0
	- `optim`: adamw_torch
	- `optim_args`: None
	- `adafactor`: False
	- `group_by_length`: False
	- `length_column_name`: length
	- `ddp_find_unused_parameters`: None
	- `ddp_bucket_cap_mb`: None
	- `ddp_broadcast_buffers`: False
	- `dataloader_pin_memory`: True
	- `dataloader_persistent_workers`: False
	- `skip_memory_metrics`: True
	- `use_legacy_prediction_loop`: False
	- `push_to_hub`: False
	- `resume_from_checkpoint`: None
	- `hub_model_id`: None
	- `hub_strategy`: every_save
	- `hub_private_repo`: False
	- `hub_always_push`: False
	- `gradient_checkpointing`: False
	- `gradient_checkpointing_kwargs`: None
	- `include_inputs_for_metrics`: False
	- `eval_do_concat_batches`: True
	- `fp16_backend`: auto
	- `push_to_hub_model_id`: None
	- `push_to_hub_organization`: None
	- `mp_parameters`:
	- `auto_find_batch_size`: False
	- `full_determinism`: False
	- `torchdynamo`: None
	- `ray_scope`: last
	- `ddp_timeout`: 1800
	- `torch_compile`: False
	- `torch_compile_backend`: None
	- `torch_compile_mode`: None
	- `dispatch_batches`: None
	- `split_batches`: None
	- `include_tokens_per_second`: False
	- `include_num_input_tokens_seen`: False
	- `neftune_noise_alpha`: None
	- `optim_target_modules`: None
	- `batch_eval_metrics`: False
	- `eval_on_start`: False
	- `eval_use_gather_object`: False
	- `batch_sampler`: no_duplicates
	- `multi_dataset_batch_sampler`: proportional

	</details>

	### Training Logs
	\| Epoch \| Step \| intfloat/multilingual-e5-small_cosine_map@100 \|
	\|:-----:\|:----:\|:---------------------------------------------:\|
	\| 0 \| 0 \| 0.875 \|


	### Framework Versions
	- Python: 3.8.10
	- Sentence Transformers: 3.0.1
	- Transformers: 4.44.2
	- PyTorch: 2.1.2+cu121
	- Accelerate: 0.32.0
	- Datasets: 2.19.1
	- Tokenizers: 0.19.1

	## Citation

	### BibTeX

	#### Sentence Transformers
	```bibtex
	@inproceedings{reimers-2019-sentence-bert,
	title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
	author = "Reimers, Nils and Gurevych, Iryna",
	booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
	month = "11",
	year = "2019",
	publisher = "Association for Computational Linguistics",
	url = "https://arxiv.org/abs/1908.10084",
	}
	```

	#### MultipleNegativesRankingLoss
	```bibtex
	@misc{henderson2017efficient,
	title={Efficient Natural Language Response Suggestion for Smart Reply},
	author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
	year={2017},
	eprint={1705.00652},
	archivePrefix={arXiv},
	primaryClass={cs.CL}
	}
	```

	<!--
	## Glossary

	Clearly define terms in order to be accessible across audiences.
	-->

	<!--
	## Model Card Authors

	Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.
	-->

	<!--
	## Model Card Contact

	Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.
	-->