langchain.callbacks.argilla_callback.ArgillaCallbackHandler¶
class langchain.callbacks.argilla_callback.ArgillaCallbackHandler(dataset_name: str, workspace_name: Optional[str] = None, api_url: Optional[str] = None, api_key: Optional[str] = None)[source]¶
Callback Handler that logs into Argilla.
Parameters
dataset_name – name of the FeedbackDataset in Argilla. Note that it must
exist in advance. If you need help on how to create a FeedbackDataset in
Argilla, please visit
https://docs.argilla.io/en/latest/guides/llms/practical_guides/use_argilla_callback_in_langchain.html.
workspace_name – name of the workspace in Argilla where the specified
FeedbackDataset lives in. Defaults to None, which means that the
default workspace will be used.
api_url – URL of the Argilla Server that we want to use, and where the
FeedbackDataset lives in. Defaults to None, which means that either
ARGILLA_API_URL environment variable or the default will be used.
api_key – API Key to connect to the Argilla Server. Defaults to None, which
means that either ARGILLA_API_KEY environment variable or the default
will be used.
Raises
ImportError – if the argilla package is not installed.
ConnectionError – if the connection to Argilla fails.
FileNotFoundError – if the FeedbackDataset retrieval from Argilla fails.
Examples
>>> from langchain.llms import OpenAI
>>> from langchain.callbacks import ArgillaCallbackHandler
>>> argilla_callback = ArgillaCallbackHandler(
...     dataset_name="my-dataset",
...     workspace_name="my-workspace",
...     api_url="http://localhost:6900",
...     api_key="argilla.apikey",
... )
>>> llm = OpenAI(
...     temperature=0,