Zero-Shot Classification
Zero-shot text classification is a task in natural language processing where a model is trained on a set of labeled examples but is then able to classify new examples from previously unseen classes.
Text Input
Dune is the best movie ever.
Candidate Labels
CINEMA, ART, MUSIC
About Zero-Shot Classification
About the Task
Zero Shot Classification is the task of predicting a class that wasn't seen by the model during training. This method, which leverages a pre-trained language model, can be thought of as an instance of transfer learning which generally refers to using a model trained for one task in a different application than what it was originally trained for. This is particularly useful for situations where the amount of labeled data is small.
In zero shot classification, we provide the model with a prompt and a sequence of text that describes what we want our model to do, in natural language. Zero-shot classification excludes any examples of the desired task being completed. This differs from single or few-shot classification, as these tasks include a single or a few examples of the selected task.
Zero, single and few-shot classification seem to be an emergent feature of large language models. This feature seems to come about around model sizes of +100M parameters. The effectiveness of a model at a zero, single or few-shot task seems to scale with model size, meaning that larger models (models with more trainable parameters or layers) generally do better at this task.
Here is an example of a zero-shot prompt for classifying the sentiment of a sequence of text:
Classify the following input text into one of the following three categories: [positive, negative, neutral]
Input Text: Hugging Face is awesome for making all of these
state of the art models available!
Sentiment: positive
One great example of this task with a nice off-the-shelf model is available at the widget of this page, where the user can input a sequence of text and candidate labels to the model. This is a word level example of zero shot classification, more elaborate and lengthy generations are available with larger models. Testing these models out and getting a feel for prompt engineering is the best way to learn how to use them.
Inference
You can use the 🤗 Transformers library zero-shot-classification pipeline to infer with zero shot text classification models.
from transformers import pipeline
pipe = pipeline(model="facebook/bart-large-mnli")
pipe("I have a problem with my iphone that needs to be resolved asap!",
candidate_labels=["urgent", "not urgent", "phone", "tablet", "computer"],
)
# output
>>> {'sequence': 'I have a problem with my iphone that needs to be resolved asap!!', 'labels': ['urgent', 'phone', 'computer', 'not urgent', 'tablet'], 'scores': [0.504, 0.479, 0.013, 0.003, 0.002]}
Useful Resources
Compatible libraries
Note Powerful zero-shot text classification model.
Note Powerful zero-shot multilingual text classification model that can accomplish multiple tasks.
Note A widely used dataset used to benchmark multiple variants of text classification.
Note The Multi-Genre Natural Language Inference (MultiNLI) corpus is a crowd-sourced collection of 433k sentence pairs annotated with textual entailment information.
Note FEVER is a publicly available dataset for fact extraction and verification against textual sources.
No example Space is defined for this task.
Note Contribute by proposing a Space for this task !
No example metric is defined for this task.
Note Contribute by proposing a metric for this task !