SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 128 tokens
Number of Classes: 14 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
6	'## How do they work?'
7	'A digital twin comprises three core elements: the physical system (product, process, network), a virtual model representing it and a data connection that updates the model in real time. The virtual model mirrors the physical system’s current state and behavior, continuously synchronized with data from sensors and internet of things devices. This setup allows the digital twin to simulate and predict the physical system’s performance under various conditions. Bringing all three components together requires several key technologies. First, the collection and use of data involves cloud computing and platforms for storage and processing. Second, AI and machine learning are needed to enable simulation models that provide advanced analytics and accurate virtual models. Lastly, augmented reality and virtual reality enable advanced visualization and interactions between the digital model and the physical system.'
9	'While data is the mantra of our modern age, data sets taken in isolation are of limited value because they tend to be sparse, noisy, and often indirect. Because systems exist across a web of components, any micro change results in a ripple effect, making accurately replicating a system extremely difficult. In banking, digital twin technology’s true potential is harnessed when integrated with a bank’s proprietary knowledge along with an inflow of external stimuli into decision-making models. With data flowing from multiple channels, using a mirrored environment enables precise contingency and incident response plans. When changes are made, other parts can adapt accordingly, simplifying coordination with business units and third parties. For example, a digital twin of a bank’s technology stack can predict outcomes of certain technology changes with the potential to evolve based on results from prior simulation runs. Digital twins can also mitigate risk across evolving fraud vectors through intelligent, comprehensive, data-driven strategic planning.'
3	'So-called “digital twins” are dynamic, virtual replicas of complex systems. Organizations often use them for scenario planning because they blend real-world elements with simulations and a constant flow of data, helping evaluate the consequences of different decisions. For example, when BMO acquired 503 Bank of the West branches in 2023, it used Matterport’s capture services to create dimensionally accurate 3D digital twins of all the branch locations within three months.'
0	'https://bankingjournal.aba.com/2024/08/precision-banking-the-digital-twin-advantage/ dt'
11	'Let’s look at a few potential use cases for banks:'
13	'Digital financial twin. This is an approach where digital twins could be used to precisely map financial and nonfinancial metrics across the life cycle of a bank product. The digital twin would be set up to link metrics related to the product’s service, partners, customers, and employees, resulting in efficient and quality decision-making. To go further, the digital twin would combine with real-time data from an enterprise resource planning system to ensure the highest level of resource optimization, drive sustainability and accelerate product development.'
10	'In the banking industry, digital twins may seem like enhanced scenario analysis. And if this is what you’re thinking, we don’t blame you. But here is where the key difference lies: data. Traditional scenario analysis relies on static data while digital twins use real-time dynamic data and facilitate bidirectional data flow. This means that a digital twin can take insights it produced and trigger changes to optimize the physical system it replicates, whereas scenario analysis merely provides an output that must be reviewed and acted upon separately.'
5	'The use of digital twins began in the 1960s when NASA used twin models to monitor and adjust spacecraft during space missions. Recently, the Biden administration announced a $285 million investment in digital twin technology for semiconductor manufacturing based on its potential to enhance efficiency, innovation, and resilience in the U.S.'
2	'img__'
4	'This technology enabled BMO to complete remote site assessments and operational tests within the model without service disruptions. The results: Over $500,000 saved in 15 months, 6,000 survey hours recouped across 503 locations, and branch resources and documentation centralized.'
12	'Stress testing. A digital twin could enable banks to simulate various scenarios, such as economic downturns, market fluctuations, or operational disruptions, to assess their resilience and performance under stress. Banks could identify weaknesses and mitigate risks preemptively by inputting diverse parameters to the digital twin. Add real-time insights and your bank can continuously adjust strategies that bolster resilience and stability.'
1	'# Precision banking: The ‘digital twin’ advantage'
8	'## What problem do digital twins solve?'

Evaluation

Metrics

Label	Accuracy
all	0.8571

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("mikeee/setfit-model")
# Run inference
preds = model("## 数字孪生解决了什么问题？")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	50.7143	156

Label	Training Sample Count
0	1
1	1
2	1
3	1
4	1
5	1
6	1
7	1
8	1
9	1
10	1
11	1
12	1
13	1

Training Hyperparameters

batch_size: (8, 8)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 4
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.1429	1	0.0039	-

Framework Versions

Python: 3.10.12
SetFit: 1.0.3
Sentence Transformers: 3.0.1
Transformers: 4.39.0
PyTorch: 2.3.1+cu121
Datasets: 2.21.0
Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

mikeee
/

setfit-model