pipeline for text to text generation using seq2seq models example python from transformers import pipeline generator pipelinemodelmrm8488t5basefinetunedquestiongenerationap generator answer manuel context manuel has created rupertabase with the support of hftransformers and google generatedtext question who created the rupertabase learn more about the basics of using a pipeline in the pipeline tutorial pipelinetutorial you can pass text generation parameters to this pipeline to control stopping criteria decoding strategy and more learn more about text generation parameters in text generation strategies generationstrategies and text generationtextgeneration this text2textgenerationpipeline pipeline can currently be loaded from pipeline using the following task identifier text2textgeneration the models that this pipeline can use are models that have been finetuned on a translation task see the uptodate list of available models on huggingface comodelshttps huggingface comodels filtertext2textgeneration for a list of available parameters see the following documentationhttps huggingface codocstransformersenmainclassestextgenerationtransformers generation generationmixin generate usage python text2textgenerator pipelinetext2textgeneration text2textgeneratorquestion what is 42 context 42 is the answer to life the universe and everything used in the return key of the pipeline returnname generated def initself args kwargs super initargs kwargs self checkmodeltype tfmodelforseqtoseqcausallmmappingnames if self framework tf else modelforseqtoseqcausallmmappingnames def sanitizeparameters self returntensorsnone returntextnone returntypenone cleanuptokenizationspacesnone truncationnone stopsequencenone generatekwargs preprocessparams if truncation is not none preprocessparamstruncation truncation forwardparams generatekwargs postprocessparams if returntensors is not none and returntype is none returntype returntype tensors if returntensors else returntype text if returntype is not none postprocessparamsreturntype returntype if cleanuptokenizationspaces is not none postprocessparamscleanuptokenizationspaces cleanuptokenizationspaces if stopsequence is not none stopsequenceids self tokenizer encodestopsequence addspecialtokensfalse if lenstopsequenceids 1 warnings warn stopping on a multiple token sequence is not yet supported on transformers the first token of the stop sequence will be used as the stop sequence string in the interim generatekwargseostokenid stopsequenceids0 return preprocessparams forwardparams postprocessparams def checkinputsself inputlength int minlength int maxlength int return true def parseandtokenizeself args truncation prefix self model config prefix if self model config prefix is not none else if isinstanceargs0 list if self tokenizer padtokenid is none raise valueerrorplease make sure that the tokenizer has a padtokenid when using a batch input args prefix arg for arg in args0 padding true elif isinstanceargs0 str args prefix args0 padding false else raise valueerror f args0 args0 have the wrong format the should be either of type str or type list inputs self tokenizerargs paddingpadding truncationtruncation returntensorsself framework this is produced by tokenizers but is an invalid generate kwargs if tokentypeids in inputs del inputstokentypeids return inputs def callself args kwargs r generate the output texts using texts given as inputs args args str or liststr input text for the encoder returntensors bool optional defaults to false whether or not to include the tensors of predictions as token indices in the outputs returntext bool optional defaults to true whether or not to include the decoded texts in the outputs cleanuptokenizationspaces bool optional defaults to false whether or not to clean up the potential extra spaces in the text output truncation truncationstrategy optional defaults to truncationstrategy donottruncate the truncation strategy for the tokenization within the pipeline truncationstrategy donottruncate default will never truncate but it is sometimes desirable to truncate the input to fit the model s maxlength instead of throwing an error down the line generatekwargs additional keyword arguments to pass along to the generate method of the model see the generate method corresponding to your framework here modelgenerativemodels return a list or a list of list of dict each result comes as a dictionary with the following keys generatedtext str present when returntexttrue the generated text generatedtokenids torch tensor or tf tensor present when returntensorstrue the token ids of the generated text summarize news articles and other documents this summarizing pipeline can currently be loaded from pipeline using the following task identifier summarization the models that this pipeline can use are models that have been finetuned on a summarization task which is currently bartlargecnn t5small t5base t5large t53b t511b see the uptodate list of available models on huggingface comodelshttps huggingface comodels filtersummarization for a list of available parameters see the following documentationhttps huggingface codocstransformersenmainclassestextgenerationtransformers generation generationmixin generate usage python use bart in pytorch summarizer pipelinesummarization summarizeran apple a day keeps the doctor away minlength5 maxlength20 use t5 in tf summarizer pipelinesummarization modelt5base tokenizert5base frameworktf summarizeran apple a day keeps the doctor away minlength5 maxlength20 used in the return key of the pipeline returnname summary def callself args kwargs r summarize the texts given as inputs args documents str or liststr one or several articles or one list of articles to summarize returntext bool optional defaults to true whether or not to include the decoded texts in the outputs returntensors bool optional defaults to false whether or not to include the tensors of predictions as token indices in the outputs cleanuptokenizationspaces bool optional defaults to false whether or not to clean up the potential extra spaces in the text output generatekwargs additional keyword arguments to pass along to the generate method of the model see the generate method corresponding to your framework here modelgenerativemodels return a list or a list of list of dict each result comes as a dictionary with the following keys summarytext str present when returntexttrue the summary of the corresponding input summarytokenids torch tensor or tf tensor present when returntensorstrue the token ids of the summary checks whether there might be something wrong with given input with regard to the model translates from one language to another this translation pipeline can currently be loaded from pipeline using the following task identifier translationxxtoyy the models that this pipeline can use are models that have been finetuned on a translation task see the uptodate list of available models on huggingface comodelshttps huggingface comodels filtertranslation for a list of available parameters see the following documentationhttps huggingface codocstransformersenmainclassestextgenerationtransformers generation generationmixin generate usage python enfrtranslator pipelinetranslationentofr enfrtranslatorhow old are you used in the return key of the pipeline returnname translation def checkinputsself inputlength int minlength int maxlength int if inputlength 0 9 maxlength logger warning fyour inputlength inputlength is bigger than 0 9 maxlength maxlength you might consider increasing your maxlength manually e g translator maxlength400 return true def preprocessself args truncationtruncationstrategy donottruncate srclangnone tgtlangnone if getattrself tokenizer buildtranslationinputs none return self tokenizer buildtranslationinputs args returntensorsself framework truncationtruncation srclangsrclang tgtlangtgtlang else return super parseandtokenizeargs truncationtruncation def sanitizeparametersself srclangnone tgtlangnone kwargs preprocessparams forwardparams postprocessparams super sanitizeparameterskwargs if srclang is not none preprocessparamssrclang srclang if tgtlang is not none preprocessparamstgtlang tgtlang if srclang is none and tgtlang is none backward compatibility direct arguments use is preferred task kwargs gettask self task items task split if task and lenitems 4 translation xx to yy preprocessparamssrclang items1 preprocessparamstgtlang items3 return preprocessparams forwardparams postprocessparams def callself args kwargs r translate the texts given as inputs args args str or liststr texts to be translated returntensors bool optional defaults to false whether or not to include the tensors of predictions as token indices in the outputs returntext bool optional defaults to true whether or not to include the decoded texts in the outputs cleanuptokenizationspaces bool optional defaults to false whether or not to clean up the potential extra spaces in the text output srclang str optional the language of the input might be required for multilingual models will not have any effect for single pair translation models tgtlang str optional the language of the desired output might be required for multilingual models will not have any effect for single pair translation models generatekwargs additional keyword arguments to pass along to the generate method of the model see the generate method corresponding to your framework here modelgenerativemodels return a list or a list of list of dict each result comes as a dictionary with the following keys translationtext str present when returntexttrue the translation translationtokenids torch tensor or tf tensor present when returntensorstrue the token ids of the translation pipeline for text to text generation using seq2seq models example python from transformers import pipeline generator pipeline model mrm8488 t5 base finetuned question generation ap generator answer manuel context manuel has created ruperta base with the support of hf transformers and google generated_text question who created the ruperta base learn more about the basics of using a pipeline in the pipeline tutorial pipeline_tutorial you can pass text generation parameters to this pipeline to control stopping criteria decoding strategy and more learn more about text generation parameters in text generation strategies generation_strategies and text generation text_generation this text2textgenerationpipeline pipeline can currently be loaded from pipeline using the following task identifier text2text generation the models that this pipeline can use are models that have been fine tuned on a translation task see the up to date list of available models on huggingface co models https huggingface co models filter text2text generation for a list of available parameters see the following documentation https huggingface co docs transformers en main_classes text_generation transformers generation generationmixin generate usage python text2text_generator pipeline text2text generation text2text_generator question what is 42 context 42 is the answer to life the universe and everything used in the return key of the pipeline checks whether there might be something wrong with given input with regard to the model this is produced by tokenizers but is an invalid generate kwargs generate the output text s using text s given as inputs args args str or list str input text for the encoder return_tensors bool optional defaults to false whether or not to include the tensors of predictions as token indices in the outputs return_text bool optional defaults to true whether or not to include the decoded texts in the outputs clean_up_tokenization_spaces bool optional defaults to false whether or not to clean up the potential extra spaces in the text output truncation truncationstrategy optional defaults to truncationstrategy do_not_truncate the truncation strategy for the tokenization within the pipeline truncationstrategy do_not_truncate default will never truncate but it is sometimes desirable to truncate the input to fit the model s max_length instead of throwing an error down the line generate_kwargs additional keyword arguments to pass along to the generate method of the model see the generate method corresponding to your framework here model generative models return a list or a list of list of dict each result comes as a dictionary with the following keys generated_text str present when return_text true the generated text generated_token_ids torch tensor or tf tensor present when return_tensors true the token ids of the generated text summarize news articles and other documents this summarizing pipeline can currently be loaded from pipeline using the following task identifier summarization the models that this pipeline can use are models that have been fine tuned on a summarization task which is currently bart large cnn t5 small t5 base t5 large t5 3b t5 11b see the up to date list of available models on huggingface co models https huggingface co models filter summarization for a list of available parameters see the following documentation https huggingface co docs transformers en main_classes text_generation transformers generation generationmixin generate usage python use bart in pytorch summarizer pipeline summarization summarizer an apple a day keeps the doctor away min_length 5 max_length 20 use t5 in tf summarizer pipeline summarization model t5 base tokenizer t5 base framework tf summarizer an apple a day keeps the doctor away min_length 5 max_length 20 used in the return key of the pipeline summarize the text s given as inputs args documents str or list str one or several articles or one list of articles to summarize return_text bool optional defaults to true whether or not to include the decoded texts in the outputs return_tensors bool optional defaults to false whether or not to include the tensors of predictions as token indices in the outputs clean_up_tokenization_spaces bool optional defaults to false whether or not to clean up the potential extra spaces in the text output generate_kwargs additional keyword arguments to pass along to the generate method of the model see the generate method corresponding to your framework here model generative models return a list or a list of list of dict each result comes as a dictionary with the following keys summary_text str present when return_text true the summary of the corresponding input summary_token_ids torch tensor or tf tensor present when return_tensors true the token ids of the summary checks whether there might be something wrong with given input with regard to the model translates from one language to another this translation pipeline can currently be loaded from pipeline using the following task identifier translation_xx_to_yy the models that this pipeline can use are models that have been fine tuned on a translation task see the up to date list of available models on huggingface co models https huggingface co models filter translation for a list of available parameters see the following documentation https huggingface co docs transformers en main_classes text_generation transformers generation generationmixin generate usage python en_fr_translator pipeline translation_en_to_fr en_fr_translator how old are you used in the return key of the pipeline backward compatibility direct arguments use is preferred translation xx to yy translate the text s given as inputs args args str or list str texts to be translated return_tensors bool optional defaults to false whether or not to include the tensors of predictions as token indices in the outputs return_text bool optional defaults to true whether or not to include the decoded texts in the outputs clean_up_tokenization_spaces bool optional defaults to false whether or not to clean up the potential extra spaces in the text output src_lang str optional the language of the input might be required for multilingual models will not have any effect for single pair translation models tgt_lang str optional the language of the desired output might be required for multilingual models will not have any effect for single pair translation models generate_kwargs additional keyword arguments to pass along to the generate method of the model see the generate method corresponding to your framework here model generative models return a list or a list of list of dict each result comes as a dictionary with the following keys translation_text str present when return_text true the translation translation_token_ids torch tensor or tf tensor present when return_tensors true the token ids of the translation
import enum
import warnings

from ..tokenization_utils import TruncationStrategy
from ..utils import add_end_docstrings, is_tf_available, is_torch_available, logging
from .base import PIPELINE_INIT_ARGS, Pipeline


if is_tf_available():
    import tensorflow as tf

    from ..models.auto.modeling_tf_auto import TF_MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING_NAMES

if is_torch_available():
    from ..models.auto.modeling_auto import MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING_NAMES

logger = logging.get_logger(__name__)


class ReturnType(enum.Enum):
    TENSORS = 0
    TEXT = 1


@add_end_docstrings(PIPELINE_INIT_ARGS)
class Text2TextGenerationPipeline(Pipeline):
    

    
    return_name = "generated"

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.check_model_type(
            TF_MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING_NAMES
            if self.framework == "tf"
            else MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING_NAMES
        )

    def _sanitize_parameters(
        self,
        return_tensors=None,
        return_text=None,
        return_type=None,
        clean_up_tokenization_spaces=None,
        truncation=None,
        stop_sequence=None,
        **generate_kwargs,
    ):
        preprocess_params = {}
        if truncation is not None:
            preprocess_params["truncation"] = truncation

        forward_params = generate_kwargs

        postprocess_params = {}
        if return_tensors is not None and return_type is None:
            return_type = ReturnType.TENSORS if return_tensors else ReturnType.TEXT
        if return_type is not None:
            postprocess_params["return_type"] = return_type

        if clean_up_tokenization_spaces is not None:
            postprocess_params["clean_up_tokenization_spaces"] = clean_up_tokenization_spaces

        if stop_sequence is not None:
            stop_sequence_ids = self.tokenizer.encode(stop_sequence, add_special_tokens=False)
            if len(stop_sequence_ids) > 1:
                warnings.warn(
                    "Stopping on a multiple token sequence is not yet supported on transformers. The first token of"
                    " the stop sequence will be used as the stop sequence string in the interim."
                )
            generate_kwargs["eos_token_id"] = stop_sequence_ids[0]

        return preprocess_params, forward_params, postprocess_params

    def check_inputs(self, input_length: int, min_length: int, max_length: int):
        
        return True

    def _parse_and_tokenize(self, *args, truncation):
        prefix = self.model.config.prefix if self.model.config.prefix is not None else ""
        if isinstance(args[0], list):
            if self.tokenizer.pad_token_id is None:
                raise ValueError("Please make sure that the tokenizer has a pad_token_id when using a batch input")
            args = ([prefix + arg for arg in args[0]],)
            padding = True

        elif isinstance(args[0], str):
            args = (prefix + args[0],)
            padding = False
        else:
            raise ValueError(
                f" `args[0]`: {args[0]} have the wrong format. The should be either of type `str` or type `list`"
            )
        inputs = self.tokenizer(*args, padding=padding, truncation=truncation, return_tensors=self.framework)
        
        if "token_type_ids" in inputs:
            del inputs["token_type_ids"]
        return inputs

    def __call__(self, *args, **kwargs):
        r

        result = super().__call__(*args, **kwargs)
        if (
            isinstance(args[0], list)
            and all(isinstance(el, str) for el in args[0])
            and all(len(res) == 1 for res in result)
        ):
            return [res[0] for res in result]
        return result

    def preprocess(self, inputs, truncation=TruncationStrategy.DO_NOT_TRUNCATE, **kwargs):
        inputs = self._parse_and_tokenize(inputs, truncation=truncation, **kwargs)
        return inputs

    def _forward(self, model_inputs, **generate_kwargs):
        if self.framework == "pt":
            in_b, input_length = model_inputs["input_ids"].shape
        elif self.framework == "tf":
            in_b, input_length = tf.shape(model_inputs["input_ids"]).numpy()

        self.check_inputs(
            input_length,
            generate_kwargs.get("min_length", self.model.config.min_length),
            generate_kwargs.get("max_length", self.model.config.max_length),
        )
        output_ids = self.model.generate(**model_inputs, **generate_kwargs)
        out_b = output_ids.shape[0]
        if self.framework == "pt":
            output_ids = output_ids.reshape(in_b, out_b // in_b, *output_ids.shape[1:])
        elif self.framework == "tf":
            output_ids = tf.reshape(output_ids, (in_b, out_b // in_b, *output_ids.shape[1:]))
        return {"output_ids": output_ids}

    def postprocess(self, model_outputs, return_type=ReturnType.TEXT, clean_up_tokenization_spaces=False):
        records = []
        for output_ids in model_outputs["output_ids"][0]:
            if return_type == ReturnType.TENSORS:
                record = {f"{self.return_name}_token_ids": output_ids}
            elif return_type == ReturnType.TEXT:
                record = {
                    f"{self.return_name}_text": self.tokenizer.decode(
                        output_ids,
                        skip_special_tokens=True,
                        clean_up_tokenization_spaces=clean_up_tokenization_spaces,
                    )
                }
            records.append(record)
        return records


@add_end_docstrings(PIPELINE_INIT_ARGS)
class SummarizationPipeline(Text2TextGenerationPipeline):
    

    
    return_name = "summary"

    def __call__(self, *args, **kwargs):
        r
        return super().__call__(*args, **kwargs)

    def check_inputs(self, input_length: int, min_length: int, max_length: int) -> bool:
        
        if max_length < min_length:
            logger.warning(f"Your min_length={min_length} must be inferior than your max_length={max_length}.")

        if input_length < max_length:
            logger.warning(
                f"Your max_length is set to {max_length}, but your input_length is only {input_length}. Since this is "
                "a summarization task, where outputs shorter than the input are typically wanted, you might "
                f"consider decreasing max_length manually, e.g. summarizer('...', max_length={input_length//2})"
            )


@add_end_docstrings(PIPELINE_INIT_ARGS)
class TranslationPipeline(Text2TextGenerationPipeline):
    

    
    return_name = "translation"

    def check_inputs(self, input_length: int, min_length: int, max_length: int):
        if input_length > 0.9 * max_length:
            logger.warning(
                f"Your input_length: {input_length} is bigger than 0.9 * max_length: {max_length}. You might consider "
                "increasing your max_length manually, e.g. translator('...', max_length=400)"
            )
        return True

    def preprocess(self, *args, truncation=TruncationStrategy.DO_NOT_TRUNCATE, src_lang=None, tgt_lang=None):
        if getattr(self.tokenizer, "_build_translation_inputs", None):
            return self.tokenizer._build_translation_inputs(
                *args, return_tensors=self.framework, truncation=truncation, src_lang=src_lang, tgt_lang=tgt_lang
            )
        else:
            return super()._parse_and_tokenize(*args, truncation=truncation)

    def _sanitize_parameters(self, src_lang=None, tgt_lang=None, **kwargs):
        preprocess_params, forward_params, postprocess_params = super()._sanitize_parameters(**kwargs)
        if src_lang is not None:
            preprocess_params["src_lang"] = src_lang
        if tgt_lang is not None:
            preprocess_params["tgt_lang"] = tgt_lang
        if src_lang is None and tgt_lang is None:
            
            task = kwargs.get("task", self.task)
            items = task.split("_")
            if task and len(items) == 4:
                
                preprocess_params["src_lang"] = items[1]
                preprocess_params["tgt_lang"] = items[3]
        return preprocess_params, forward_params, postprocess_params

    def __call__(self, *args, **kwargs):
        r
        return super().__call__(*args, **kwargs)