langchain.smith.evaluation.config.RunEvalConfig¶
class langchain.smith.evaluation.config.RunEvalConfig[source]¶
Bases: BaseModel
Configuration for a run evaluation.
Parameters
evaluators (List[Union[EvaluatorType, EvalConfig]]) – Configurations for which evaluators to apply to the dataset run.
Each can be the string of an EvaluatorType, such
as EvaluatorType.QA, the evaluator type string (“qa”), or a configuration for a
given evaluator (e.g., RunEvalConfig.QA).
custom_evaluators (Optional[List[Union[RunEvaluator, StringEvaluator]]]) – Custom evaluators to apply to the dataset run.
reference_key (Optional[str]) – The key in the dataset run to use as the reference string.
If not provided, it will be inferred automatically.
prediction_key (Optional[str]) – The key from the traced run’s outputs dictionary to use to
represent the prediction. If not provided, it will be inferred
automatically.
input_key (Optional[str]) – The key from the traced run’s inputs dictionary to use to represent the
input. If not provided, it will be inferred automatically.
eval_llm (Optional[BaseLanguageModel]) – The language model to pass to any evaluators that use a language model.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param custom_evaluators: Optional[List[Union[langsmith.evaluation.evaluator.RunEvaluator, langchain.evaluation.schema.StringEvaluator]]] = None¶
Custom evaluators to apply to the dataset run.
param eval_llm: Optional[langchain.schema.language_model.BaseLanguageModel] = None¶
The language model to pass to any evaluators that require one.