langchain.callbacks.labelstudio_callback.LabelStudioCallbackHandler¶
class langchain.callbacks.labelstudio_callback.LabelStudioCallbackHandler(api_key: Optional[str] = None, url: Optional[str] = None, project_id: Optional[int] = None, project_name: str = 'LangChain-%Y-%m-%d', project_config: Optional[str] = None, mode: Union[str, LabelStudioMode] = LabelStudioMode.PROMPT)[source]¶
Label Studio callback handler.
Provides the ability to send predictions to Label Studio
for human evaluation, feedback and annotation.
Parameters
api_key – Label Studio API key
url – Label Studio URL
project_id – Label Studio project ID
project_name – Label Studio project name
project_config – Label Studio project config (XML)
mode – Label Studio mode (“prompt” or “chat”)
Examples
>>> from langchain.llms import OpenAI
>>> from langchain.callbacks import LabelStudioCallbackHandler
>>> handler = LabelStudioCallbackHandler(
...             api_key='<your_key_here>',
...             url='http://localhost:8080',
...             project_name='LangChain-%Y-%m-%d',
...             mode='prompt'
... )
>>> llm = OpenAI(callbacks=[handler])
>>> llm.predict('Tell me a story about a dog.')
Attributes
DEFAULT_PROJECT_NAME
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
ignore_retry
Whether to ignore retry callbacks.
raise_error
run_inline
Methods
__init__([api_key, url, project_id, ...])
add_prompts_generations(run_id, generations)