langchain.callbacks.confident_callback.DeepEvalCallbackHandler¶
class langchain.callbacks.confident_callback.DeepEvalCallbackHandler(metrics: List[Any], implementation_name: Optional[str] = None)[source]¶
Callback Handler that logs into deepeval.
Parameters
implementation_name – name of the implementation in deepeval
metrics – A list of metrics
Raises
ImportError – if the deepeval package is not installed.
Examples
>>> from langchain.llms import OpenAI
>>> from langchain.callbacks import DeepEvalCallbackHandler
>>> from deepeval.metrics import AnswerRelevancy
>>> metric = AnswerRelevancy(minimum_score=0.3)
>>> deepeval_callback = DeepEvalCallbackHandler(
...     implementation_name="exampleImplementation",
...     metrics=[metric],
... )
>>> llm = OpenAI(
...     temperature=0,
...     callbacks=[deepeval_callback],
...     verbose=True,
...     openai_api_key="API_KEY_HERE",
... )
>>> llm.generate([
...     "What is the best evaluation tool out there? (no bias at all)",
... ])
"Deepeval, no doubt about it."
Initializes the deepevalCallbackHandler.
Parameters
implementation_name – Name of the implementation you want.
metrics – What metrics do you want to track?
Raises
ImportError – if the deepeval package is not installed.
ConnectionError – if the connection to deepeval fails.
Attributes
BLOG_URL
ISSUES_URL
REPO_URL
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
ignore_retry
Whether to ignore retry callbacks.