langchain.evaluation.comparison.eval_chain.PairwiseStringEvalChain¶
class langchain.evaluation.comparison.eval_chain.PairwiseStringEvalChain[source]¶
Bases: PairwiseStringEvaluator, LLMEvalChain, LLMChain
A chain for comparing two outputs, such as the outputsof two models, prompts, or outputs of a single model on similar inputs.
output_parser¶
The output parser for the chain.
Type
BaseOutputParser
Example
>>> from langchain.chat_models import ChatOpenAI
>>> from langchain.evaluation.comparison import PairwiseStringEvalChain
>>> llm = ChatOpenAI(temperature=0)
>>> chain = PairwiseStringEvalChain.from_llm(llm=llm)
>>> result = chain.evaluate_string_pairs(
...     input = "What is the chemical formula for water?",
...     prediction = "H2O",
...     prediction_b = (
...        "The chemical formula for water is H2O, which means"
...        " there are two hydrogen atoms and one oxygen atom."
...     reference = "The chemical formula for water is H2O.",
... )
>>> print(result["text"])
# {
#    "value": "B",
#    "comment": "Both responses accurately state"
#       " that the chemical formula for water is H2O."
#       " However, Response B provides additional information"
# .     " by explaining what the formula means.\n[[B]]"
# }
param callback_manager: Optional[BaseCallbackManager] = None¶
Deprecated, use callbacks instead.
param callbacks: Callbacks = None¶
Optional list of callback handlers (or callback manager). Defaults to None.
Callback handlers are called throughout the lifecycle of a call to a chain,