SetFit

This is a SetFit model that can be used for Text Classification. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 2 classes
Language: en
License: apache-2.0

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
negative	"definitely not for kids i've just taken 2 nieces, 9 and 14, to see barbie. they both hated it... really, really hated it!! and, me: i'm just stunned. i've no idea what i've just seen.if barbie is meant to be a dark satire, it's alright. if it's meant to be a kids movie, it's unconscionably bad. and, i'm totally confused which type of movie it's supposed to be. what the literal heck is going on!!if you're looking for a twenty first century dark cultural satire, you're at least in your twenties, you've an open mind, this is your movie... maybe. if you're looking for a movie to take your kids to, before getting ice cream, this is absolutely not the movie you're looking for. my advice: give barbie a miss and go straight for the ice cream. you will be so much happier. i wish i had. 15 out of 30 found this helpful. was this review helpful? sign in to vote. permalink" "the most political movie i've ever seen. i loved the effort with the sets and the fun outfits. i also really enjoyed the ads threw out movie.i was really enjoying this movie but then it started getting really political, i was kind of fine with it but then they started making politics the whole story line and so it was really no longer a fun movie. after the movie finished i just ended up leaving miserable and like i waisted money... how is a family meant to enjoy this?? i can't see any 13 year old understanding anything they talk about.also they want to talk about how being a human is hard and that were not perfect but then they hire someone to play a teen girl who has perfect hair and skin. a bit of a confusing message? 180 out of 276 found this helpful. was this review helpful? sign in to vote. permalink" "mediocre at best i had high expectations as a result of the media press, however these we're crushed from the start.first of all, i did expect it to feel a little generic and cheesy although all i could feel throughout the movie is that it was rushed and cheap. the storyline was dry and over-political.i'd have liked to see more lgbtq+ representation and breaking stereotypes, as it felt like just a generic film. even the barbies were considered stereotypes!comedy throughout was limited and wasn't engaging at all - sometimes trying too hard to emphasise feminism and going too far in the opposite direction to the world is heading in (i'm not encouraging patriarchy).however, i did find the music quite good (especially the billie eilish song) and did redeem the movie a little bit.in summary, the movie was mediocre at best and there's not much to discuss. it felt like a cheap version of movies like spirited and the greatest showman with no real emotion."
positive	"fabulously done so, i didn't have that much of a high expectation going in to see this because i really wasn't sure what is was going to be. i grew up playing barbie's and have watched the animated films many times with my kids.the movie from start to finish is so thoughtfully executed. it's for everyone, but mainly young girls need to see this. it's a great reminder of how special we are as women and how special all of us are individually as a person and the life we get to experience.another fantastic element is adding a very 90s feel to the film. you will laugh like you used to laugh when fun comedies would come out in theatre's years ago. i haven't seen just a good comedy with heart in years.i brought my 7yr old and we dressed up for it too. i wanted her to see me comfortable in my own skin and her as well and the film touches on that. it's suited for any age, regardless of the rating.i don't give high ratings like this to films usually, as i find small things usually that i wish had been addressed more... but not in barbie. what a fun movie and a great time you'll have watching this in theatre (or at home) knowing it's your childhood come to life." "sure, life in plastic is fantastic but only in barbieland the barbie movie was unexpected. it was a colorful and clever ride in the cinema. i was not expecting all the deep emotions that the film gave. life is not all about perfection, which a lot of people wants to constantly achieve these days, imperfections is what identifies us and that's okay.i'm a visual kind of cinema goer and i have the say that dance sequence (dance the night) and the scene where barbie and ken where traveling to the real world were amazing. kudos to the creative team behind it, i absolutely adore the set.greta gerwig and her team delivered a visual spectacle with a heart. 10 out of 28 found this helpful. was this review helpful? sign in to vote. permalink" "funny with social commentary barbie (margot robbie) lives in barbieland. ken (ryan gosling) is always desperate for her attention. all the ken and barbie dolls live in a happy matriarchy unaware of the real world. out of the blue, barbie starts pondering death and her existence. weird barbie (kate mckinnon) tells her that she has opened a portal to the real world and must fix whatever is her real world problem before she can return to her perfect self. she and stowaway ken find the real world completely different than their expectations. mattel ceo (will ferrell) insists on putting her back in the box. mattel secretary gloria (america ferrera) had caused the problem by drawing new forms of barbie.first, i love the premise and i love the 2001 opening. the start is a lot funnier than i expected. margot robbie is great and ryan gosling is hilarious. it's a great start in barbieland. the first moments in la is a little too much. quite frankly, barbie and ken would not stand out at venice beach. the male leering is good enough. they don't need to do the stereotype abusive guys. the real world should be realistic. barbie and ken would be better fishes out of the water in a recognizable regular world. the more average the real world is, the better it is for the characters to showcase their outrageousness.mattel is fun and i like the ridiculous silliness. will ferrell is a good way to show their outrageous quality. the movie has a lot of social commentary and that aspect does threaten to overwhelm it. the last act has some wonky moments as the movie tries to wrap its arms around the heavier social discussions. i would like to keep that more contained and concentrate on the mother daughter relationships. it's gloria and sasha, but it's also ruth and barbara. that's a perfect way to end the movie. i do like the montage idea, but i didn't realize what it was doing initially. it would have been fine to do that for the closing credits. all in all, this is a funny engaging movie and it's able to deal with some of the tougher social material."

Label

Examples

negative

"definitely not for kids i've just taken 2 nieces, 9 and 14, to see barbie. they both hated it... really, really hated it!! and, me: i'm just stunned. i've no idea what i've just seen.if barbie is meant to be a dark satire, it's alright. if it's meant to be a kids movie, it's unconscionably bad. and, i'm totally confused which type of movie it's supposed to be. what the literal heck is going on!!if you're looking for a twenty first century dark cultural satire, you're at least in your twenties, you've an open mind, this is your movie... maybe. if you're looking for a movie to take your kids to, before getting ice cream, this is absolutely not the movie you're looking for. my advice: give barbie a miss and go straight for the ice cream. you will be so much happier. i wish i had. 15 out of 30 found this helpful. was this review helpful? sign in to vote. permalink"
"the most political movie i've ever seen. i loved the effort with the sets and the fun outfits. i also really enjoyed the ads threw out movie.i was really enjoying this movie but then it started getting really political, i was kind of fine with it but then they started making politics the whole story line and so it was really no longer a fun movie. after the movie finished i just ended up leaving miserable and like i waisted money... how is a family meant to enjoy this?? i can't see any 13 year old understanding anything they talk about.also they want to talk about how being a human is hard and that were not perfect but then they hire someone to play a teen girl who has perfect hair and skin. a bit of a confusing message? 180 out of 276 found this helpful. was this review helpful? sign in to vote. permalink"
"mediocre at best i had high expectations as a result of the media press, however these we're crushed from the start.first of all, i did expect it to feel a little generic and cheesy although all i could feel throughout the movie is that it was rushed and cheap. the storyline was dry and over-political.i'd have liked to see more lgbtq+ representation and breaking stereotypes, as it felt like just a generic film. even the barbies were considered stereotypes!comedy throughout was limited and wasn't engaging at all - sometimes trying too hard to emphasise feminism and going too far in the opposite direction to the world is heading in (i'm not encouraging patriarchy).however, i did find the music quite good (especially the billie eilish song) and did redeem the movie a little bit.in summary, the movie was mediocre at best and there's not much to discuss. it felt like a cheap version of movies like spirited and the greatest showman with no real emotion."

positive

"fabulously done so, i didn't have that much of a high expectation going in to see this because i really wasn't sure what is was going to be. i grew up playing barbie's and have watched the animated films many times with my kids.the movie from start to finish is so thoughtfully executed. it's for everyone, but mainly young girls need to see this. it's a great reminder of how special we are as women and how special all of us are individually as a person and the life we get to experience.another fantastic element is adding a very 90s feel to the film. you will laugh like you used to laugh when fun comedies would come out in theatre's years ago. i haven't seen just a good comedy with heart in years.i brought my 7yr old and we dressed up for it too. i wanted her to see me comfortable in my own skin and her as well and the film touches on that. it's suited for any age, regardless of the rating.i don't give high ratings like this to films usually, as i find small things usually that i wish had been addressed more... but not in barbie. what a fun movie and a great time you'll have watching this in theatre (or at home) knowing it's your childhood come to life."
"sure, life in plastic is fantastic but only in barbieland the barbie movie was unexpected. it was a colorful and clever ride in the cinema. i was not expecting all the deep emotions that the film gave. life is not all about perfection, which a lot of people wants to constantly achieve these days, imperfections is what identifies us and that's okay.i'm a visual kind of cinema goer and i have the say that dance sequence (dance the night) and the scene where barbie and ken where traveling to the real world were amazing. kudos to the creative team behind it, i absolutely adore the set.greta gerwig and her team delivered a visual spectacle with a heart. 10 out of 28 found this helpful. was this review helpful? sign in to vote. permalink"
"funny with social commentary barbie (margot robbie) lives in barbieland. ken (ryan gosling) is always desperate for her attention. all the ken and barbie dolls live in a happy matriarchy unaware of the real world. out of the blue, barbie starts pondering death and her existence. weird barbie (kate mckinnon) tells her that she has opened a portal to the real world and must fix whatever is her real world problem before she can return to her perfect self. she and stowaway ken find the real world completely different than their expectations. mattel ceo (will ferrell) insists on putting her back in the box. mattel secretary gloria (america ferrera) had caused the problem by drawing new forms of barbie.first, i love the premise and i love the 2001 opening. the start is a lot funnier than i expected. margot robbie is great and ryan gosling is hilarious. it's a great start in barbieland. the first moments in la is a little too much. quite frankly, barbie and ken would not stand out at venice beach. the male leering is good enough. they don't need to do the stereotype abusive guys. the real world should be realistic. barbie and ken would be better fishes out of the water in a recognizable regular world. the more average the real world is, the better it is for the characters to showcase their outrageousness.mattel is fun and i like the ridiculous silliness. will ferrell is a good way to show their outrageous quality. the movie has a lot of social commentary and that aspect does threaten to overwhelm it. the last act has some wonky moments as the movie tries to wrap its arms around the heavier social discussions. i would like to keep that more contained and concentrate on the mother daughter relationships. it's gloria and sasha, but it's also ruth and barbara. that's a perfect way to end the movie. i do like the montage idea, but i didn't realize what it was doing initially. it would have been fine to do that for the closing credits. all in all, this is a funny engaging movie and it's able to deal with some of the tougher social material."

Evaluation

Metrics

Label	Accuracy	Precision	Recall	F1
all	0.8812	0.9952	0.8757	0.9316

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("carlesoctav/SentimentClassifierBarbieDune-8shot")
# Run inference
preds = model("decent i like what they did with this movie and the characters with its combining the barbie world and the real world. barbie starts getting \"vibes\" and has to go into the real world to find the girl who played with her to set things right and winds up in the mattel headquarters. something resembling chaos ensues. ken joins her and winds up causing further damage. i like what they do in various stages of the story and with the characters. it was overall a very pleasant surprise snd a good movie with a good cast. margot robbie, ryan gosling, america ferrera, and will ferrell were all good in their roles. if you are a movie and/or a barbie fan, you will love this movie.*** out of **** 2 out of 7 found this helpful. was this review helpful? sign in to vote. permalink")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	112	234.1953	1424

Label	Training Sample Count
negative	64
positive	64

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0019	1	0.3627	-
0.0962	50	0.0007	-
0.1923	100	0.1003	-
0.2885	150	0.0001	-
0.3846	200	0.0001	-
0.4808	250	0.0001	-
0.5769	300	0.0001	-
0.6731	350	0.0	-
0.7692	400	0.0001	-
0.8654	450	0.0	-
0.9615	500	0.0	-
1.0	520	-	0.2312

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.11
SetFit: 1.0.3
Sentence Transformers: 2.5.1
Transformers: 4.38.2
PyTorch: 2.0.1
Datasets: 2.18.0
Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

carlesoctav
/

SentimentClassifierBarbieDune-8shot