langchain.evaluation.schema.AgentTrajectoryEvaluator¶
class langchain.evaluation.schema.AgentTrajectoryEvaluator[source]¶
Interface for evaluating agent trajectories.
Attributes
requires_input
Whether this evaluator requires an input string.
requires_reference
Whether this evaluator requires a reference label.
Methods
__init__()
aevaluate_agent_trajectory(*, prediction, ...)
Asynchronously evaluate a trajectory.
evaluate_agent_trajectory(*, prediction, ...)
Evaluate a trajectory.
__init__()¶
async aevaluate_agent_trajectory(*, prediction: str, agent_trajectory: Sequence[Tuple[AgentAction, str]], input: str, reference: Optional[str] = None, **kwargs: Any) → dict[source]¶
Asynchronously evaluate a trajectory.
Parameters
prediction (str) – The final predicted response.
agent_trajectory (List[Tuple[AgentAction, str]]) – The intermediate steps forming the agent trajectory.
input (str) – The input to the agent.
reference (Optional[str]) – The reference answer.
Returns
The evaluation result.
Return type
dict
evaluate_agent_trajectory(*, prediction: str, agent_trajectory: Sequence[Tuple[AgentAction, str]], input: str, reference: Optional[str] = None, **kwargs: Any) → dict[source]¶
Evaluate a trajectory.
Parameters
prediction (str) – The final predicted response.
agent_trajectory (List[Tuple[AgentAction, str]]) – The intermediate steps forming the agent trajectory.
input (str) – The input to the agent.
reference (Optional[str]) – The reference answer.
Returns
The evaluation result.
Return type
dict
Examples using AgentTrajectoryEvaluator¶
Custom Trajectory Evaluator