The language model to pass to any evaluators that require one.
param evaluators: List[Union[langchain.evaluation.schema.EvaluatorType, str, langchain.smith.evaluation.config.EvalConfig]] [Optional]¶
Configurations for which evaluators to apply to the dataset run.
Each can be the string of an
EvaluatorType, such
as EvaluatorType.QA, the evaluator type string (“qa”), or a configuration for a
given evaluator
(e.g.,
RunEvalConfig.QA).
param input_key: Optional[str] = None¶
The key from the traced run’s inputs dictionary to use to represent the
input. If not provided, it will be inferred automatically.
param prediction_key: Optional[str] = None¶
The key from the traced run’s outputs dictionary to use to
represent the prediction. If not provided, it will be inferred
automatically.
param reference_key: Optional[str] = None¶
The key in the dataset run to use as the reference string.
If not provided, we will attempt to infer automatically.
class CoTQA[source]¶
Bases: EvalConfig
Configuration for a context-based QA evaluator.
Parameters
prompt (Optional[BasePromptTemplate]) – The prompt template to use for generating the question.
llm (Optional[BaseLanguageModel]) – The language model to use for the evaluation chain.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param evaluator_type: langchain.evaluation.schema.EvaluatorType = EvaluatorType.CONTEXT_QA¶
param llm: Optional[langchain.schema.language_model.BaseLanguageModel] = None¶
param prompt: Optional[langchain.schema.prompt_template.BasePromptTemplate] = None¶