langchain.evaluation.schema.StringEvaluator¶
class langchain.evaluation.schema.StringEvaluator[source]¶
Grade, tag, or otherwise evaluate predictions relative to their inputs
and/or reference labels.
Attributes
evaluation_name
The name of the evaluation.
requires_input
Whether this evaluator requires an input string.
requires_reference
Whether this evaluator requires a reference label.
Methods
__init__()
aevaluate_strings(*, prediction[, ...])
Asynchronously evaluate Chain or LLM output, based on optional input and label.
evaluate_strings(*, prediction[, reference, ...])
Evaluate Chain or LLM output, based on optional input and label.
__init__()¶
async aevaluate_strings(*, prediction: str, reference: Optional[str] = None, input: Optional[str] = None, **kwargs: Any) → dict[source]¶
Asynchronously evaluate Chain or LLM output, based on optional input and label.
Parameters
prediction (str) – The LLM or chain prediction to evaluate.
reference (Optional[str], optional) – The reference label to evaluate against.
input (Optional[str], optional) – The input to consider during evaluation.
**kwargs – Additional keyword arguments, including callbacks, tags, etc.
Returns
The evaluation results containing the score or value.
Return type
dict
evaluate_strings(*, prediction: str, reference: Optional[str] = None, input: Optional[str] = None, **kwargs: Any) → dict[source]¶
Evaluate Chain or LLM output, based on optional input and label.
Parameters
prediction (str) – The LLM or chain prediction to evaluate.
reference (Optional[str], optional) – The reference label to evaluate against.
input (Optional[str], optional) – The input to consider during evaluation.
**kwargs – Additional keyword arguments, including callbacks, tags, etc.
Returns