utility class containing a conversation and its history this class is meant to be used as an input to the conversationalpipeline the conversation contains several utility functions to manage the addition of new user inputs and generated model responses arguments messages unionstr listdictstr str optional the initial messages to start the conversation either a string or a list of dicts containing role and content keys if a string is passed it is interpreted as a single message with the user role conversationid uuid uuid optional unique identifier for the conversation if not provided a random uuid4 id will be assigned to the conversation usage python conversation conversationgoing to the movies tonight any suggestions conversation addmessagerole assistant content the big lebowski conversation addmessagerole user content is it good def init self messages unionstr listdictstr str none conversationid uuid uuid none deprecatedkwargs if not conversationid conversationid uuid uuid4 if messages is none text deprecatedkwargs poptext none if text is not none messages role user content text else messages elif isinstancemessages str messages role user content messages this block deals with the legacy args new code should just totally avoid pastuserinputs and generatedresponses self numprocesseduserinputs 0 generatedresponses deprecatedkwargs popgeneratedresponses none pastuserinputs deprecatedkwargs poppastuserinputs none if generatedresponses is not none and pastuserinputs is none raise valueerrorgeneratedresponses cannot be passed without pastuserinputs if pastuserinputs is not none legacymessages if generatedresponses is none generatedresponses we structure it this way instead of using zip because the lengths may differ by 1 for i in rangemaxlenpastuserinputs lengeneratedresponses if i lenpastuserinputs legacymessages appendrole user content pastuserinputsi if i lengeneratedresponses legacymessages appendrole assistant content generatedresponsesi messages legacymessages messages self uuid conversationid self messages messages def eqself other if not isinstanceother conversation return false return self uuid other uuid or self messages other messages def addmessageself message dictstr str if not setmessage keys role content raise valueerrormessage should contain only role and content keys if messagerole not in user assistant system raise valueerroronly user assistant and system roles are supported for now self messages appendmessage def adduserinputself text str overwrite bool false if lenself 0 and self1role user if overwrite logger warning f user input added while unprocessed input was existing self1content was overwritten f with text self1content text else logger warning f user input added while unprocessed input was existing self1content new input f ignored text set overwrite to true to overwrite unprocessed user input else self messages appendrole user content text def appendresponseself response str self messages appendrole assistant content response def markprocessedself self numprocesseduserinputs lenself usermessages def iterself for message in self messages yield message def getitemself item return self messagesitem def setitemself key value self messageskey value def lenself return lenself messages def reprself output fconversation id self uuidn for message in self messages output fmessage role message content n return output def itertextsself this is a legacy method for backwards compatibility it is recommended to just directly access conversation messages instead for message in self messages yield messagerole user messagecontent property def usermessagesself this is a legacy property for backwards compatibility it is recommended to just directly access conversation messages instead return messagecontent for message in self messages if messagerole user property def pastuserinputsself this is a legacy property for backwards compatibility it is recommended to just directly access conversation messages instead the modern class does not care about which messages are processed or not if not self usermessages return in the past the most recent user message had to be markprocessed before being included in pastusermessages the class essentially had a singlemessage buffer representing messages that had not yet been replied to this is no longer the case but we mimic the behaviour in this property for backward compatibility if self messages1role user or self numprocesseduserinputs lenself usermessages return self usermessages return self usermessages 1 property def generatedresponsesself this is a legacy property for backwards compatibility it is recommended to just directly access conversation messages instead return messagecontent for message in self messages if messagerole assistant property def newuserinputself this is a legacy property for backwards compatibility it is recommended to just directly access conversation messages instead return self usermessages1 addenddocstrings pipelineinitargs r minlengthforresponse int optional defaults to 32 the minimum length in number of tokens for a response minimumtokens int optional defaults to 10 the minimum length of tokens to leave for a response multiturn conversational pipeline example python from transformers import pipeline conversation any model with a chat template can be used in a conversationalpipeline chatbot pipelinemodelfacebookblenderbot400mdistill conversation objects initialized with a string will treat it as a user message conversation conversationi m looking for a movie what s your favourite one conversation chatbotconversation conversation messages1content i don t really have a favorite movie but i do like action movies what about you conversation addmessagerole user content that s interesting why do you like action movies conversation chatbotconversation conversation messages1content i think it s just because they re so fastpaced and actionfantastic learn more about the basics of using a pipeline in the pipeline tutorial pipelinetutorial this conversational pipeline can currently be loaded from pipeline using the following task identifier conversational this pipeline can be used with any model that has a chat templatehttps huggingface codocstransformerschattemplating set self maxlength generatekwargs getmaxlength self model config maxlength xxx numworkers0 is required to be backward compatible otherwise the threads will require a conversation copy this will definitely hinder performance on gpu but has to be opted in because of this bc change if isinstanceconversations list and isinstanceconversations0 dict conversations conversationconversations elif isinstanceconversations list and isinstanceconversations0 list conversations conversationconv for conv in conversations outputs super callconversations numworkersnumworkers kwargs if isinstanceoutputs list and lenoutputs 1 return outputs0 return outputs def preprocessself conversation conversation minlengthforresponse32 dictstr any inputids self tokenizer applychattemplateconversation addgenerationprompttrue if self framework pt inputids torch longtensorinputids elif self framework tf inputids tf constantinputids return inputids inputids conversation conversation def forwardself modelinputs minimumtokens10 generatekwargs n modelinputsinputids shape1 conversation modelinputs popconversation if maxlength not in generatekwargs and maxnewtokens not in generatekwargs generatekwargsmaxnewtokens 256 outputids self model generatemodelinputs generatekwargs if self model config isencoderdecoder startposition 1 else startposition n return outputids outputids startposition conversation conversation def postprocessself modeloutputs cleanuptokenizationspacestrue outputids modeloutputsoutputids answer self tokenizer decode outputids0 skipspecialtokenstrue cleanuptokenizationspacescleanuptokenizationspaces conversation modeloutputsconversation conversation addmessagerole assistant content answer return conversation utility class containing a conversation and its history this class is meant to be used as an input to the conversationalpipeline the conversation contains several utility functions to manage the addition of new user inputs and generated model responses arguments messages union str list dict str str optional the initial messages to start the conversation either a string or a list of dicts containing role and content keys if a string is passed it is interpreted as a single message with the user role conversation_id uuid uuid optional unique identifier for the conversation if not provided a random uuid4 id will be assigned to the conversation usage python conversation conversation going to the movies tonight any suggestions conversation add_message role assistant content the big lebowski conversation add_message role user content is it good this block deals with the legacy args new code should just totally avoid past_user_inputs and generated_responses we structure it this way instead of using zip because the lengths may differ by 1 add a user input to the conversation for the next round this is a legacy method that assumes that inputs must alternate user assistant user assistant and so will not add multiple user messages in succession we recommend just using add_message with role user instead this is a legacy method we recommend just using add_message with an appropriate role instead this is a legacy method as the conversation no longer distinguishes between processed and unprocessed user input we set a counter here to keep behaviour mostly backward compatible but in general you should just read the messages directly when writing new code generates a string representation of the conversation returns str example conversation id 7d15686b dc94 49f2 9c4b c9eac6a1f114 user going to the movies tonight any suggestions bot the big lebowski this is a legacy method for backwards compatibility it is recommended to just directly access conversation messages instead this is a legacy property for backwards compatibility it is recommended to just directly access conversation messages instead this is a legacy property for backwards compatibility it is recommended to just directly access conversation messages instead the modern class does not care about which messages are processed or not in the past the most recent user message had to be mark_processed before being included in past_user_messages the class essentially had a single message buffer representing messages that had not yet been replied to this is no longer the case but we mimic the behaviour in this property for backward compatibility this is a legacy property for backwards compatibility it is recommended to just directly access conversation messages instead this is a legacy property for backwards compatibility it is recommended to just directly access conversation messages instead min_length_for_response int optional defaults to 32 the minimum length in number of tokens for a response minimum_tokens int optional defaults to 10 the minimum length of tokens to leave for a response multi turn conversational pipeline example python from transformers import pipeline conversation any model with a chat template can be used in a conversationalpipeline chatbot pipeline model facebook blenderbot 400m distill conversation objects initialized with a string will treat it as a user message conversation conversation i m looking for a movie what s your favourite one conversation chatbot conversation conversation messages 1 content i don t really have a favorite movie but i do like action movies what about you conversation add_message role user content that s interesting why do you like action movies conversation chatbot conversation conversation messages 1 content i think it s just because they re so fast paced and action fantastic learn more about the basics of using a pipeline in the pipeline tutorial pipeline_tutorial this conversational pipeline can currently be loaded from pipeline using the following task identifier conversational this pipeline can be used with any model that has a chat template https huggingface co docs transformers chat_templating set self max_length generate_kwargs get max_length self model config max_length generate responses for the conversation s given as inputs args conversations a conversation or a list of conversation conversation to generate responses for inputs can also be passed as a list of dictionaries with role and content keys in this case they will be converted to conversation objects automatically multiple conversations in either format may be passed as a list clean_up_tokenization_spaces bool optional defaults to false whether or not to clean up the potential extra spaces in the text output generate_kwargs additional keyword arguments to pass along to the generate method of the model see the generate method corresponding to your framework here model generative models returns conversation or a list of conversation conversation s with updated generated responses for those containing a new user input xxx num_workers 0 is required to be backward compatible otherwise the threads will require a conversation copy this will definitely hinder performance on gpu but has to be opted in because of this bc change
import uuid
from typing import Any, Dict, List, Union

from ..utils import add_end_docstrings, is_tf_available, is_torch_available, logging
from .base import PIPELINE_INIT_ARGS, Pipeline


if is_tf_available():
    import tensorflow as tf

if is_torch_available():
    import torch


logger = logging.get_logger(__name__)


class Conversation:
    

    def __init__(
        self, messages: Union[str, List[Dict[str, str]]] = None, conversation_id: uuid.UUID = None, **deprecated_kwargs
    ):
        if not conversation_id:
            conversation_id = uuid.uuid4()

        if messages is None:
            text = deprecated_kwargs.pop("text", None)
            if text is not None:
                messages = [{"role": "user", "content": text}]
            else:
                messages = []
        elif isinstance(messages, str):
            messages = [{"role": "user", "content": messages}]

        
        
        self._num_processed_user_inputs = 0
        generated_responses = deprecated_kwargs.pop("generated_responses", None)
        past_user_inputs = deprecated_kwargs.pop("past_user_inputs", None)
        if generated_responses is not None and past_user_inputs is None:
            raise ValueError("generated_responses cannot be passed without past_user_inputs!")
        if past_user_inputs is not None:
            legacy_messages = []
            if generated_responses is None:
                generated_responses = []
            
            for i in range(max([len(past_user_inputs), len(generated_responses)])):
                if i < len(past_user_inputs):
                    legacy_messages.append({"role": "user", "content": past_user_inputs[i]})
                if i < len(generated_responses):
                    legacy_messages.append({"role": "assistant", "content": generated_responses[i]})
            messages = legacy_messages + messages

        self.uuid = conversation_id
        self.messages = messages

    def __eq__(self, other):
        if not isinstance(other, Conversation):
            return False
        return self.uuid == other.uuid or self.messages == other.messages

    def add_message(self, message: Dict[str, str]):
        if not set(message.keys()) == {"role", "content"}:
            raise ValueError("Message should contain only 'role' and 'content' keys!")
        if message["role"] not in ("user", "assistant", "system"):
            raise ValueError("Only 'user', 'assistant' and 'system' roles are supported for now!")
        self.messages.append(message)

    def add_user_input(self, text: str, overwrite: bool = False):
        
        if len(self) > 0 and self[-1]["role"] == "user":
            if overwrite:
                logger.warning(
                    f'User input added while unprocessed input was existing: "{self[-1]["content"]}" was overwritten '
                    f'with: "{text}".'
                )
                self[-1]["content"] = text
            else:
                logger.warning(
                    f'User input added while unprocessed input was existing: "{self[-1]["content"]}" new input '
                    f'ignored: "{text}". Set `overwrite` to True to overwrite unprocessed user input'
                )
        else:
            self.messages.append({"role": "user", "content": text})

    def append_response(self, response: str):
        
        self.messages.append({"role": "assistant", "content": response})

    def mark_processed(self):
        
        self._num_processed_user_inputs = len(self._user_messages)

    def __iter__(self):
        for message in self.messages:
            yield message

    def __getitem__(self, item):
        return self.messages[item]

    def __setitem__(self, key, value):
        self.messages[key] = value

    def __len__(self):
        return len(self.messages)

    def __repr__(self):
        
        output = f"Conversation id: {self.uuid}\n"
        for message in self.messages:
            output += f"{message['role']}: {message['content']}\n"
        return output

    def iter_texts(self):
        
        
        for message in self.messages:
            yield message["role"] == "user", message["content"]

    @property
    def _user_messages(self):
        
        
        return [message["content"] for message in self.messages if message["role"] == "user"]

    @property
    def past_user_inputs(self):
        
        
        
        if not self._user_messages:
            return []
        
        
        
        
        if self.messages[-1]["role"] != "user" or self._num_processed_user_inputs == len(self._user_messages):
            return self._user_messages

        return self._user_messages[:-1]

    @property
    def generated_responses(self):
        
        
        return [message["content"] for message in self.messages if message["role"] == "assistant"]

    @property
    def new_user_input(self):
        
        
        return self._user_messages[-1]


@add_end_docstrings(
    PIPELINE_INIT_ARGS,
    r,
)
class ConversationalPipeline(Pipeline):
    

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        if self.tokenizer.pad_token_id is None:
            self.tokenizer.pad_token = self.tokenizer.eos_token

    def _sanitize_parameters(
        self, min_length_for_response=None, minimum_tokens=None, clean_up_tokenization_spaces=None, **generate_kwargs
    ):
        preprocess_params = {}
        forward_params = {}
        postprocess_params = {}

        if min_length_for_response is not None:
            preprocess_params["min_length_for_response"] = min_length_for_response
        if minimum_tokens is not None:
            forward_params["minimum_tokens"] = minimum_tokens

        if "max_length" in generate_kwargs:
            forward_params["max_length"] = generate_kwargs["max_length"]
            
        if clean_up_tokenization_spaces is not None:
            postprocess_params["clean_up_tokenization_spaces"] = clean_up_tokenization_spaces

        if generate_kwargs:
            forward_params.update(generate_kwargs)
        return preprocess_params, forward_params, postprocess_params

    def __call__(self, conversations: Union[List[Dict], Conversation, List[Conversation]], num_workers=0, **kwargs):
        r
        
        
        
        
        if isinstance(conversations, list) and isinstance(conversations[0], dict):
            conversations = Conversation(conversations)
        elif isinstance(conversations, list) and isinstance(conversations[0], list):
            conversations = [Conversation(conv) for conv in conversations]
        outputs = super().__call__(conversations, num_workers=num_workers, **kwargs)
        if isinstance(outputs, list) and len(outputs) == 1:
            return outputs[0]
        return outputs

    def preprocess(self, conversation: Conversation, min_length_for_response=32) -> Dict[str, Any]:
        input_ids = self.tokenizer.apply_chat_template(conversation, add_generation_prompt=True)

        if self.framework == "pt":
            input_ids = torch.LongTensor([input_ids])
        elif self.framework == "tf":
            input_ids = tf.constant([input_ids])
        return {"input_ids": input_ids, "conversation": conversation}

    def _forward(self, model_inputs, minimum_tokens=10, **generate_kwargs):
        n = model_inputs["input_ids"].shape[1]
        conversation = model_inputs.pop("conversation")
        if "max_length" not in generate_kwargs and "max_new_tokens" not in generate_kwargs:
            generate_kwargs["max_new_tokens"] = 256
        output_ids = self.model.generate(**model_inputs, **generate_kwargs)
        if self.model.config.is_encoder_decoder:
            start_position = 1
        else:
            start_position = n
        return {"output_ids": output_ids[:, start_position:], "conversation": conversation}

    def postprocess(self, model_outputs, clean_up_tokenization_spaces=True):
        output_ids = model_outputs["output_ids"]
        answer = self.tokenizer.decode(
            output_ids[0],
            skip_special_tokens=True,
            clean_up_tokenization_spaces=clean_up_tokenization_spaces,
        )
        conversation = model_outputs["conversation"]
        conversation.add_message({"role": "assistant", "content": answer})
        return conversation