Run the chain as text in, text out or multiple variables, text out.
save(file_path: Union[Path, str]) → None¶
Save the chain.
Parameters
file_path – Path to file to save the chain to.
Example:
.. code-block:: python
chain.save(file_path=”path/chain.yaml”)
validator set_verbose  »  verbose¶
If verbose is None, set it.
This allows users to pass in None as verbose to access the global setting.
to_json() → Union[SerializedConstructor, SerializedNotImplemented]¶
to_json_not_implemented() → SerializedNotImplemented¶
property lc_attributes: Dict¶
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]¶
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]¶
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool¶
Return whether or not the class is serializable.
model Config¶
Bases: object
Configuration for this pydantic object.
arbitrary_types_allowed = True¶
extra = 'forbid'¶


langchain.evaluation.run_evaluators.implementations.get_criteria_evaluator¶
langchain.evaluation.run_evaluators.implementations.get_criteria_evaluator(llm: BaseLanguageModel, criteria: Union[Mapping[str, str], Sequence[str], str], *, input_key: str = 'input', prediction_key: str = 'output', prompt: BasePromptTemplate = PromptTemplate(input_variables=['input', 'output', 'criteria'], output_parser=None, partial_variables={}, template='You are assessing a submitted answer on a given task or input based on a set of criteria. Here is the data:\n[BEGIN DATA]\n***\n[Task]: {input}\n***\n[Submission]: {output}\n***\n[Criteria]: {criteria}\n***\n[END DATA]\nDoes the submission meet all the Criteria? First, write out in a step by step manner your reasoning about each criterion to be sure that your conclusion is correct. Avoid simply stating the correct answers at the outset. Then print only the single character "Y" or "N" (without quotes or punctuation) on its own line corresponding to the correct answer of whether the submission meets all criteria. At the end, repeat just the letter again by itself on a new line.', template_format='f-string', validate_template=True), evaluation_name: Optional[str] = None, **kwargs: Any) → RunEvaluatorChain[source]¶
Get an eval chain for grading a model’s response against a map of criteria.


langchain.evaluation.run_evaluators.base.RunEvaluatorChain¶
class langchain.evaluation.run_evaluators.base.RunEvaluatorChain(*, memory: Optional[BaseMemory] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, callback_manager: Optional[BaseCallbackManager] = None, verbose: bool = None, tags: Optional[List[str]] = None, input_mapper: RunEvaluatorInputMapper, eval_chain: Chain, output_parser: RunEvaluatorOutputParser)[source]¶
Bases: Chain, RunEvaluator
Evaluate Run and optional examples.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param callback_manager: Optional[BaseCallbackManager] = None¶
Deprecated, use callbacks instead.
param callbacks: Callbacks = None¶
Optional list of callback handlers (or callback manager). Defaults to None.
Callback handlers are called throughout the lifecycle of a call to a chain,
starting with on_chain_start, ending with on_chain_end or on_chain_error.
Each custom chain can optionally call additional callback methods, see Callback docs
for full details.
param eval_chain: Chain [Required]¶
The evaluation chain.
param input_mapper: RunEvaluatorInputMapper [Required]¶
Maps the Run and Optional example to a dictionary for the eval chain.
param memory: Optional[BaseMemory] = None¶
Optional memory object. Defaults to None.
Memory is a class that gets called at the start
and at the end of every chain. At the start, memory loads variables and passes
them along in the chain. At the end, it saves any returned variables.
There are many different types of memory - please see memory docs
for the full catalog.
param output_parser: RunEvaluatorOutputParser [Required]¶


for the full catalog.
param output_parser: RunEvaluatorOutputParser [Required]¶
Parse the output of the eval chain into feedback.
param tags: Optional[List[str]] = None¶
Optional list of tags associated with the chain. Defaults to None
These tags will be associated with each call to this chain,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a chain with its use case.
param verbose: bool [Optional]¶
Whether or not run in verbose mode. In verbose mode, some intermediate logs
will be printed to the console. Defaults to langchain.verbose value.
__call__(inputs: Union[Dict[str, Any], Any], return_only_outputs: bool = False, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, include_run_info: bool = False) → Dict[str, Any]¶
Run the logic of this chain and add to output if desired.
Parameters
inputs – Dictionary of inputs, or single input if chain expects
only one param.
return_only_outputs – boolean for whether to return only outputs in the
response. If True, only new keys generated by this chain will be
returned. If False, both input keys and new keys generated by this
chain will be returned. Defaults to False.
callbacks – Callbacks to use for this chain run. If not provided, will
use the callbacks provided to the chain.
include_run_info – Whether to include run info in the response. Defaults
to False.


include_run_info – Whether to include run info in the response. Defaults
to False.
async acall(inputs: Union[Dict[str, Any], Any], return_only_outputs: bool = False, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, include_run_info: bool = False) → Dict[str, Any]¶
Run the logic of this chain and add to output if desired.
Parameters
inputs – Dictionary of inputs, or single input if chain expects
only one param.
return_only_outputs – boolean for whether to return only outputs in the
response. If True, only new keys generated by this chain will be
returned. If False, both input keys and new keys generated by this
chain will be returned. Defaults to False.
callbacks – Callbacks to use for this chain run. If not provided, will
use the callbacks provided to the chain.
include_run_info – Whether to include run info in the response. Defaults
to False.
async aevaluate_run(run: Run, example: Optional[Example] = None) → EvaluationResult[source]¶
Evaluate an example.
apply(input_list: List[Dict[str, Any]], callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → List[Dict[str, str]]¶
Call the chain on all inputs in the list.
async arun(*args: Any, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, tags: Optional[List[str]] = None, **kwargs: Any) → str¶
Run the chain as text in, text out or multiple variables, text out.
dict(**kwargs: Any) → Dict¶
Return dictionary representation of chain.
evaluate_run(run: Run, example: Optional[Example] = None) → EvaluationResult[source]¶


Evaluate an example.
prep_inputs(inputs: Union[Dict[str, Any], Any]) → Dict[str, str]¶
Validate and prep inputs.
prep_outputs(inputs: Dict[str, str], outputs: Dict[str, str], return_only_outputs: bool = False) → Dict[str, str]¶
Validate and prep outputs.
validator raise_deprecation  »  all fields¶
Raise deprecation warning if callback_manager is used.
run(*args: Any, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, tags: Optional[List[str]] = None, **kwargs: Any) → str¶
Run the chain as text in, text out or multiple variables, text out.
save(file_path: Union[Path, str]) → None¶
Save the chain.
Parameters
file_path – Path to file to save the chain to.
Example:
.. code-block:: python
chain.save(file_path=”path/chain.yaml”)
validator set_verbose  »  verbose¶
If verbose is None, set it.
This allows users to pass in None as verbose to access the global setting.
to_json() → Union[SerializedConstructor, SerializedNotImplemented]¶
to_json_not_implemented() → SerializedNotImplemented¶
property input_keys: List[str]¶
Input keys this chain expects.
property lc_attributes: Dict¶
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]¶
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]¶
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}


eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool¶
Return whether or not the class is serializable.
property output_keys: List[str]¶
Output keys this chain expects.
model Config¶
Bases: object
Configuration for this pydantic object.
arbitrary_types_allowed = True¶


langchain.evaluation.qa.eval_chain.ContextQAEvalChain¶
class langchain.evaluation.qa.eval_chain.ContextQAEvalChain(*, memory: Optional[BaseMemory] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, callback_manager: Optional[BaseCallbackManager] = None, verbose: bool = None, tags: Optional[List[str]] = None, prompt: BasePromptTemplate, llm: BaseLanguageModel, output_key: str = 'text', output_parser: BaseLLMOutputParser = None, return_final_only: bool = True, llm_kwargs: dict = None)[source]¶
Bases: LLMChain
LLM Chain specifically for evaluating QA w/o GT based on context
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param callback_manager: Optional[BaseCallbackManager] = None¶
Deprecated, use callbacks instead.
param callbacks: Callbacks = None¶
Optional list of callback handlers (or callback manager). Defaults to None.
Callback handlers are called throughout the lifecycle of a call to a chain,
starting with on_chain_start, ending with on_chain_end or on_chain_error.
Each custom chain can optionally call additional callback methods, see Callback docs
for full details.
param llm: BaseLanguageModel [Required]¶
Language model to call.
param llm_kwargs: dict [Optional]¶
param memory: Optional[BaseMemory] = None¶
Optional memory object. Defaults to None.
Memory is a class that gets called at the start
and at the end of every chain. At the start, memory loads variables and passes
them along in the chain. At the end, it saves any returned variables.
There are many different types of memory - please see memory docs
for the full catalog.


There are many different types of memory - please see memory docs
for the full catalog.
param output_key: str = 'text'¶
param output_parser: BaseLLMOutputParser [Optional]¶
Output parser to use.
Defaults to one that takes the most likely string but does not change it
otherwise.
param prompt: BasePromptTemplate [Required]¶
Prompt object to use.
param return_final_only: bool = True¶
Whether to return only the final parsed result. Defaults to True.
If false, will return a bunch of extra information about the generation.
param tags: Optional[List[str]] = None¶
Optional list of tags associated with the chain. Defaults to None
These tags will be associated with each call to this chain,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a chain with its use case.
param verbose: bool [Optional]¶
Whether or not run in verbose mode. In verbose mode, some intermediate logs
will be printed to the console. Defaults to langchain.verbose value.
__call__(inputs: Union[Dict[str, Any], Any], return_only_outputs: bool = False, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, include_run_info: bool = False) → Dict[str, Any]¶
Run the logic of this chain and add to output if desired.
Parameters
inputs – Dictionary of inputs, or single input if chain expects
only one param.
return_only_outputs – boolean for whether to return only outputs in the
response. If True, only new keys generated by this chain will be
returned. If False, both input keys and new keys generated by this
chain will be returned. Defaults to False.


chain will be returned. Defaults to False.
callbacks – Callbacks to use for this chain run. If not provided, will
use the callbacks provided to the chain.
include_run_info – Whether to include run info in the response. Defaults
to False.
async aapply(input_list: List[Dict[str, Any]], callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → List[Dict[str, str]]¶
Utilize the LLM generate method for speed gains.
async aapply_and_parse(input_list: List[Dict[str, Any]], callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → Sequence[Union[str, List[str], Dict[str, str]]]¶
Call apply and then parse the results.
async acall(inputs: Union[Dict[str, Any], Any], return_only_outputs: bool = False, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, include_run_info: bool = False) → Dict[str, Any]¶
Run the logic of this chain and add to output if desired.
Parameters
inputs – Dictionary of inputs, or single input if chain expects
only one param.
return_only_outputs – boolean for whether to return only outputs in the
response. If True, only new keys generated by this chain will be
returned. If False, both input keys and new keys generated by this
chain will be returned. Defaults to False.
callbacks – Callbacks to use for this chain run. If not provided, will
use the callbacks provided to the chain.
include_run_info – Whether to include run info in the response. Defaults
to False.


include_run_info – Whether to include run info in the response. Defaults
to False.
async aevaluate_strings(*, prediction: str, reference: Optional[str] = None, input: Optional[str] = None, **kwargs: Any) → dict[source]¶
async agenerate(input_list: List[Dict[str, Any]], run_manager: Optional[AsyncCallbackManagerForChainRun] = None) → LLMResult¶
Generate LLM result from inputs.
apply(input_list: List[Dict[str, Any]], callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → List[Dict[str, str]]¶
Utilize the LLM generate method for speed gains.
apply_and_parse(input_list: List[Dict[str, Any]], callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → Sequence[Union[str, List[str], Dict[str, str]]]¶
Call apply and then parse the results.
async apredict(callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, **kwargs: Any) → str¶
Format prompt with kwargs and pass to LLM.
Parameters
callbacks – Callbacks to pass to LLMChain
**kwargs – Keys to pass to prompt template.
Returns
Completion from LLM.
Example
completion = llm.predict(adjective="funny")
async apredict_and_parse(callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, **kwargs: Any) → Union[str, List[str], Dict[str, str]]¶
Call apredict and then parse the results.
async aprep_prompts(input_list: List[Dict[str, Any]], run_manager: Optional[AsyncCallbackManagerForChainRun] = None) → Tuple[List[PromptValue], Optional[List[str]]]¶
Prepare prompts from inputs.


Prepare prompts from inputs.
async arun(*args: Any, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, tags: Optional[List[str]] = None, **kwargs: Any) → str¶
Run the chain as text in, text out or multiple variables, text out.
create_outputs(llm_result: LLMResult) → List[Dict[str, Any]]¶
Create outputs from response.
dict(**kwargs: Any) → Dict¶
Return dictionary representation of chain.
evaluate(examples: List[dict], predictions: List[dict], question_key: str = 'query', context_key: str = 'context', prediction_key: str = 'result', *, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → List[dict][source]¶
Evaluate question answering examples and predictions.
evaluate_strings(*, prediction: str, reference: Optional[str] = None, input: Optional[str] = None, **kwargs: Any) → dict[source]¶


classmethod from_llm(llm: BaseLanguageModel, prompt: PromptTemplate = PromptTemplate(input_variables=['query', 'context', 'result'], output_parser=None, partial_variables={}, template="You are a teacher grading a quiz.\nYou are given a question, the context the question is about, and the student's answer. You are asked to score the student's answer as either CORRECT or INCORRECT, based on the context.\n\nExample Format:\nQUESTION: question here\nCONTEXT: context the question is about here\nSTUDENT ANSWER: student's answer here\nGRADE: CORRECT or INCORRECT here\n\nGrade the student answers based ONLY on their factual accuracy. Ignore differences in punctuation and phrasing between the student answer and true answer. It is OK if the student answer contains more information than the true answer, as long as it does not contain any conflicting statements. Begin! \n\nQUESTION: {query}\nCONTEXT: {context}\nSTUDENT ANSWER: {result}\nGRADE:", template_format='f-string', validate_template=True), **kwargs: Any) → ContextQAEvalChain[source]¶
Load QA Eval Chain from LLM.
Parameters
llm (BaseLanguageModel) – the base language model to use.
prompt ('context' and 'result' that will be used as the) – A prompt template containing the input_variables:
'query' –
prompt –
evaluation. (for) –
PROMPT. (Defaults to) –
**kwargs – additional keyword arguments.
Returns
the loaded QA eval chain.
Return type
ContextQAEvalChain
classmethod from_string(llm: BaseLanguageModel, template: str) → LLMChain¶
Create LLMChain from LLM and template.


Create LLMChain from LLM and template.
generate(input_list: List[Dict[str, Any]], run_manager: Optional[CallbackManagerForChainRun] = None) → LLMResult¶
Generate LLM result from inputs.
predict(callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, **kwargs: Any) → str¶
Format prompt with kwargs and pass to LLM.
Parameters
callbacks – Callbacks to pass to LLMChain
**kwargs – Keys to pass to prompt template.
Returns
Completion from LLM.
Example
completion = llm.predict(adjective="funny")
predict_and_parse(callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, **kwargs: Any) → Union[str, List[str], Dict[str, Any]]¶
Call predict and then parse the results.
prep_inputs(inputs: Union[Dict[str, Any], Any]) → Dict[str, str]¶
Validate and prep inputs.
prep_outputs(inputs: Dict[str, str], outputs: Dict[str, str], return_only_outputs: bool = False) → Dict[str, str]¶
Validate and prep outputs.
prep_prompts(input_list: List[Dict[str, Any]], run_manager: Optional[CallbackManagerForChainRun] = None) → Tuple[List[PromptValue], Optional[List[str]]]¶
Prepare prompts from inputs.
validator raise_deprecation  »  all fields¶
Raise deprecation warning if callback_manager is used.
run(*args: Any, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, tags: Optional[List[str]] = None, **kwargs: Any) → str¶
Run the chain as text in, text out or multiple variables, text out.
save(file_path: Union[Path, str]) → None¶
Save the chain.
Parameters


Save the chain.
Parameters
file_path – Path to file to save the chain to.
Example:
.. code-block:: python
chain.save(file_path=”path/chain.yaml”)
validator set_verbose  »  verbose¶
If verbose is None, set it.
This allows users to pass in None as verbose to access the global setting.
to_json() → Union[SerializedConstructor, SerializedNotImplemented]¶
to_json_not_implemented() → SerializedNotImplemented¶
property lc_attributes: Dict¶
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]¶
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]¶
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool¶
Return whether or not the class is serializable.
model Config¶
Bases: object
Configuration for this pydantic object.
arbitrary_types_allowed = True¶
extra = 'forbid'¶


langchain.evaluation.run_evaluators.implementations.get_qa_evaluator¶
langchain.evaluation.run_evaluators.implementations.get_qa_evaluator(llm: BaseLanguageModel, *, prompt: Union[PromptTemplate, str] = PromptTemplate(input_variables=['query', 'result', 'answer'], output_parser=None, partial_variables={}, template="You are a teacher grading a quiz.\nYou are given a question, the student's answer, and the true answer, and are asked to score the student answer as either CORRECT or INCORRECT.\n\nExample Format:\nQUESTION: question here\nSTUDENT ANSWER: student's answer here\nTRUE ANSWER: true answer here\nGRADE: CORRECT or INCORRECT here\n\nGrade the student answers based ONLY on their factual accuracy. Ignore differences in punctuation and phrasing between the student answer and true answer. It is OK if the student answer contains more information than the true answer, as long as it does not contain any conflicting statements. Begin! \n\nQUESTION: {query}\nSTUDENT ANSWER: {result}\nTRUE ANSWER: {answer}\nGRADE:", template_format='f-string', validate_template=True), input_key: str = 'input', prediction_key: str = 'output', answer_key: str = 'output', evaluation_name: Optional[str] = None, **kwargs: Any) → RunEvaluatorChain[source]¶
Get an eval chain that compares response against ground truth.


langchain.callbacks.wandb_callback.WandbCallbackHandler¶
class langchain.callbacks.wandb_callback.WandbCallbackHandler(job_type: Optional[str] = None, project: Optional[str] = 'langchain_callback_demo', entity: Optional[str] = None, tags: Optional[Sequence] = None, group: Optional[str] = None, name: Optional[str] = None, notes: Optional[str] = None, visualize: bool = False, complexity_metrics: bool = False, stream_logs: bool = False)[source]¶
Bases: BaseMetadataCallbackHandler, BaseCallbackHandler
Callback Handler that logs to Weights and Biases.
Parameters
job_type (str) – The type of job.
project (str) – The project to log to.
entity (str) – The entity to log to.
tags (list) – The tags to log.
group (str) – The group to log to.
name (str) – The name of the run.
notes (str) – The notes to log.
visualize (bool) – Whether to visualize the run.
complexity_metrics (bool) – Whether to log complexity metrics.
stream_logs (bool) – Whether to stream callback actions to W&B
This handler will utilize the associated callback method called and formats
the input of each callback function with metadata regarding the state of LLM run,
and adds the response to the list of records for both the {method}_records and
action. It then logs the response using the run.log() method to Weights and Biases.
Initialize callback handler.
Methods
__init__([job_type, project, entity, tags, ...])
Initialize callback handler.
flush_tracker([langchain_asset, reset, ...])
Flush the tracker and reset the session.
get_custom_callback_meta()


Flush the tracker and reset the session.
get_custom_callback_meta()
on_agent_action(action, **kwargs)
Run on agent action.
on_agent_finish(finish, **kwargs)
Run when agent ends running.
on_chain_end(outputs, **kwargs)
Run when chain ends running.
on_chain_error(error, **kwargs)
Run when chain errors.
on_chain_start(serialized, inputs, **kwargs)
Run when chain starts running.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, **kwargs)
Run when LLM ends running.
on_llm_error(error, **kwargs)
Run when LLM errors.
on_llm_new_token(token, **kwargs)
Run when LLM generates a new token.
on_llm_start(serialized, prompts, **kwargs)
Run when LLM starts.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.
on_text(text, **kwargs)
Run when agent is ending.
on_tool_end(output, **kwargs)
Run when tool ends running.
on_tool_error(error, **kwargs)
Run when tool errors.
on_tool_start(serialized, input_str, **kwargs)
Run when tool starts running.
reset_callback_meta()
Reset the callback metadata.
Attributes
always_verbose
Whether to call verbose callbacks even if verbose is False.
ignore_agent
Whether to ignore agent callbacks.
ignore_chain


ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
flush_tracker(langchain_asset: Any = None, reset: bool = True, finish: bool = False, job_type: Optional[str] = None, project: Optional[str] = None, entity: Optional[str] = None, tags: Optional[Sequence] = None, group: Optional[str] = None, name: Optional[str] = None, notes: Optional[str] = None, visualize: Optional[bool] = None, complexity_metrics: Optional[bool] = None) → None[source]¶
Flush the tracker and reset the session.
Parameters
langchain_asset – The langchain asset to save.
reset – Whether to reset the session.
finish – Whether to finish the run.
job_type – The job type.
project – The project.
entity – The entity.
tags – The tags.
group – The group.
name – The name.
notes – The notes.
visualize – Whether to visualize.
complexity_metrics – Whether to compute complexity metrics.
Returns – None
get_custom_callback_meta() → Dict[str, Any]¶
on_agent_action(action: AgentAction, **kwargs: Any) → Any[source]¶
Run on agent action.
on_agent_finish(finish: AgentFinish, **kwargs: Any) → None[source]¶
Run when agent ends running.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain ends running.
on_chain_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when chain errors.


Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain starts running.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when LLM errors.
on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Run when LLM generates a new token.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]¶
Run when LLM starts.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.
on_text(text: str, **kwargs: Any) → None[source]¶
Run when agent is ending.


Run when agent is ending.
on_tool_end(output: str, **kwargs: Any) → None[source]¶
Run when tool ends running.
on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, **kwargs: Any) → None[source]¶
Run when tool starts running.
reset_callback_meta() → None¶
Reset the callback metadata.
property always_verbose: bool¶
Whether to call verbose callbacks even if verbose is False.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.manager.env_var_is_set¶
langchain.callbacks.manager.env_var_is_set(env_var: str) → bool[source]¶
Check if an environment variable is set.
Parameters
env_var (str) – The name of the environment variable.
Returns
True if the environment variable is set, False otherwise.
Return type
bool


langchain.callbacks.streaming_stdout.StreamingStdOutCallbackHandler¶
class langchain.callbacks.streaming_stdout.StreamingStdOutCallbackHandler[source]¶
Bases: BaseCallbackHandler
Callback handler for streaming. Only works with LLMs that support streaming.
Methods
__init__()
on_agent_action(action, **kwargs)
Run on agent action.
on_agent_finish(finish, **kwargs)
Run on agent end.
on_chain_end(outputs, **kwargs)
Run when chain ends running.
on_chain_error(error, **kwargs)
Run when chain errors.
on_chain_start(serialized, inputs, **kwargs)
Run when chain starts running.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, **kwargs)
Run when LLM ends running.
on_llm_error(error, **kwargs)
Run when LLM errors.
on_llm_new_token(token, **kwargs)
Run on new LLM token.
on_llm_start(serialized, prompts, **kwargs)
Run when LLM starts running.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.
on_text(text, **kwargs)
Run on arbitrary text.
on_tool_end(output, **kwargs)
Run when tool ends running.
on_tool_error(error, **kwargs)
Run when tool errors.
on_tool_start(serialized, input_str, **kwargs)
Run when tool starts running.
Attributes


Run when tool starts running.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, **kwargs: Any) → Any[source]¶
Run on agent action.
on_agent_finish(finish: AgentFinish, **kwargs: Any) → None[source]¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain ends running.
on_chain_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain starts running.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when LLM errors.
on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Run on new LLM token. Only available when streaming is enabled.


Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]¶
Run when LLM starts running.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.
on_text(text: str, **kwargs: Any) → None[source]¶
Run on arbitrary text.
on_tool_end(output: str, **kwargs: Any) → None[source]¶
Run when tool ends running.
on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, **kwargs: Any) → None[source]¶
Run when tool starts running.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.mlflow_callback.analyze_text¶
langchain.callbacks.mlflow_callback.analyze_text(text: str, nlp: Any = None) → dict[source]¶
Analyze text using textstat and spacy.
Parameters
text (str) – The text to analyze.
nlp (spacy.lang) – The spacy language model to use for visualization.
Returns
A dictionary containing the complexity metrics and visualizationfiles serialized to  HTML string.
Return type
(dict)


langchain.callbacks.clearml_callback.import_clearml¶
langchain.callbacks.clearml_callback.import_clearml() → Any[source]¶
Import the clearml python package and raise an error if it is not installed.


langchain.callbacks.tracers.langchain_v1.LangChainTracerV1¶
class langchain.callbacks.tracers.langchain_v1.LangChainTracerV1(**kwargs: Any)[source]¶
Bases: BaseTracer
An implementation of the SharedTracer that POSTS to the langchain endpoint.
Initialize the LangChain tracer.
Methods
__init__(**kwargs)
Initialize the LangChain tracer.
load_default_session()
Load the default tracing session and set it as the Tracer's session.
load_session(session_name)
Load a session with the given name from the tracer.
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id, **kwargs)
End a trace for a chain run.
on_chain_error(error, *, run_id, **kwargs)
Handle an error for a chain run.
on_chain_start(serialized, inputs, *, run_id)
Start a trace for a chain run.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id, **kwargs)
End a trace for an LLM run.
on_llm_error(error, *, run_id, **kwargs)
Handle an error for an LLM run.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Start a trace for an LLM run.
on_retriever_end(documents, *, run_id, **kwargs)


on_retriever_end(documents, *, run_id, **kwargs)
Run when Retriever ends running.
on_retriever_error(error, *, run_id, **kwargs)
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id, **kwargs)
End a trace for a tool run.
on_tool_error(error, *, run_id, **kwargs)
Handle an error for a tool run.
on_tool_start(serialized, input_str, *, run_id)
Start a trace for a tool run.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
load_default_session() → Union[TracerSessionV1, TracerSession][source]¶
Load the default tracing session and set it as the Tracer’s session.
load_session(session_name: str) → Union[TracerSessionV1, TracerSession][source]¶
Load a session with the given name from the tracer.
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.


Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a chain run.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a chain run.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for a chain run.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for an LLM run.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for an LLM run.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for an LLM run.


Start a trace for an LLM run.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a tool run.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a tool run.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for a tool run.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶
run_map: Dict[str, langchain.callbacks.tracers.schemas.Run]¶


langchain.callbacks.streamlit.streamlit_callback_handler.LLMThoughtState¶
class langchain.callbacks.streamlit.streamlit_callback_handler.LLMThoughtState(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]¶
Bases: Enum
Attributes
THINKING
RUNNING_TOOL
COMPLETE
COMPLETE = 'COMPLETE'¶
RUNNING_TOOL = 'RUNNING_TOOL'¶
THINKING = 'THINKING'¶


langchain.callbacks.tracers.stdout.ConsoleCallbackHandler¶
class langchain.callbacks.tracers.stdout.ConsoleCallbackHandler(**kwargs: Any)[source]¶
Bases: BaseTracer
Tracer that prints to the console.
Methods
__init__(**kwargs)
get_breadcrumbs(run)
get_parents(run)
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id, **kwargs)
End a trace for a chain run.
on_chain_error(error, *, run_id, **kwargs)
Handle an error for a chain run.
on_chain_start(serialized, inputs, *, run_id)
Start a trace for a chain run.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id, **kwargs)
End a trace for an LLM run.
on_llm_error(error, *, run_id, **kwargs)
Handle an error for an LLM run.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Start a trace for an LLM run.
on_retriever_end(documents, *, run_id, **kwargs)
Run when Retriever ends running.
on_retriever_error(error, *, run_id, **kwargs)
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.


Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id, **kwargs)
End a trace for a tool run.
on_tool_error(error, *, run_id, **kwargs)
Handle an error for a tool run.
on_tool_start(serialized, input_str, *, run_id)
Start a trace for a tool run.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
name
raise_error
run_inline
get_breadcrumbs(run: Run) → str[source]¶
get_parents(run: Run) → List[Run][source]¶
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a chain run.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a chain run.


Handle an error for a chain run.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for a chain run.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for an LLM run.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for an LLM run.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for an LLM run.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever errors.


Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a tool run.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a tool run.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for a tool run.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
name = 'console_callback_handler'¶
raise_error: bool = False¶
run_inline: bool = False¶
run_map: Dict[str, Run]¶


langchain.callbacks.tracers.schemas.TracerSessionV1Create¶
class langchain.callbacks.tracers.schemas.TracerSessionV1Create(*, start_time: datetime = None, name: Optional[str] = None, extra: Optional[Dict[str, Any]] = None)[source]¶
Bases: TracerSessionV1Base
Create class for TracerSessionV1.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param extra: Optional[Dict[str, Any]] = None¶
param name: Optional[str] = None¶
param start_time: datetime.datetime [Optional]¶


langchain.callbacks.base.BaseCallbackHandler¶
class langchain.callbacks.base.BaseCallbackHandler[source]¶
Bases: LLMManagerMixin, ChainManagerMixin, ToolManagerMixin, RetrieverManagerMixin, CallbackManagerMixin, RunManagerMixin
Base callback handler that can be used to handle callbacks from langchain.
Methods
__init__()
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id[, parent_run_id])
Run when chain ends running.
on_chain_error(error, *, run_id[, parent_run_id])
Run when chain errors.
on_chain_start(serialized, inputs, *, run_id)
Run when chain starts running.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id[, parent_run_id])
Run when LLM ends running.
on_llm_error(error, *, run_id[, parent_run_id])
Run when LLM errors.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Run when LLM starts running.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.


Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id[, parent_run_id])
Run when tool ends running.
on_tool_error(error, *, run_id[, parent_run_id])
Run when tool errors.
on_tool_start(serialized, input_str, *, run_id)
Run when tool starts running.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when chain ends running.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when chain starts running.


Run when chain starts running.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM errors.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when LLM starts running.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶


Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when tool ends running.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when tool starts running.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.comet_ml_callback.import_comet_ml¶
langchain.callbacks.comet_ml_callback.import_comet_ml() → Any[source]¶


langchain.callbacks.tracers.evaluation.EvaluatorCallbackHandler¶
class langchain.callbacks.tracers.evaluation.EvaluatorCallbackHandler(evaluators: Sequence[RunEvaluator], max_workers: Optional[int] = None, client: Optional[LangChainPlusClient] = None, example_id: Optional[Union[UUID, str]] = None, **kwargs: Any)[source]¶
Bases: BaseTracer
A tracer that runs a run evaluator whenever a run is persisted.
Parameters
evaluators (Sequence[RunEvaluator]) – The run evaluators to apply to all top level runs.
max_workers (int, optional) – The maximum number of worker threads to use for running the evaluators.
If not specified, it will default to the number of evaluators.
client (LangChainPlusClient, optional) – The LangChainPlusClient instance to use for evaluating the runs.
If not specified, a new instance will be created.
example_id (Union[UUID, str], optional) – The example ID to be associated with the runs.
example_id¶
The example ID associated with the runs.
Type
Union[UUID, None]
client¶
The LangChainPlusClient instance used for evaluating the runs.
Type
LangChainPlusClient
evaluators¶
The sequence of run evaluators to be executed.
Type
Sequence[RunEvaluator]
executor¶
The thread pool executor used for running the evaluators.
Type
ThreadPoolExecutor
futures¶
The set of futures representing the running evaluators.
Type
Set[Future]
Methods
__init__(evaluators[, max_workers, client, ...])
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.


Run on agent end.
on_chain_end(outputs, *, run_id, **kwargs)
End a trace for a chain run.
on_chain_error(error, *, run_id, **kwargs)
Handle an error for a chain run.
on_chain_start(serialized, inputs, *, run_id)
Start a trace for a chain run.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id, **kwargs)
End a trace for an LLM run.
on_llm_error(error, *, run_id, **kwargs)
Handle an error for an LLM run.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Start a trace for an LLM run.
on_retriever_end(documents, *, run_id, **kwargs)
Run when Retriever ends running.
on_retriever_error(error, *, run_id, **kwargs)
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id, **kwargs)
End a trace for a tool run.
on_tool_error(error, *, run_id, **kwargs)
Handle an error for a tool run.
on_tool_start(serialized, input_str, *, run_id)
Start a trace for a tool run.
wait_for_futures()


Start a trace for a tool run.
wait_for_futures()
Wait for all futures to complete.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
name
raise_error
run_inline
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a chain run.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a chain run.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for a chain run.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, **kwargs: Any) → None¶


End a trace for an LLM run.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for an LLM run.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for an LLM run.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a tool run.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a tool run.


Handle an error for a tool run.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for a tool run.
wait_for_futures() → None[source]¶
Wait for all futures to complete.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
name = 'evaluator_callback_handler'¶
raise_error: bool = False¶
run_inline: bool = False¶
run_map: Dict[str, Run]¶


langchain.callbacks.file.FileCallbackHandler¶
class langchain.callbacks.file.FileCallbackHandler(filename: str, mode: str = 'a', color: Optional[str] = None)[source]¶
Bases: BaseCallbackHandler
Callback Handler that writes to a file.
Initialize callback handler.
Methods
__init__(filename[, mode, color])
Initialize callback handler.
on_agent_action(action[, color])
Run on agent action.
on_agent_finish(finish[, color])
Run on agent end.
on_chain_end(outputs, **kwargs)
Print out that we finished a chain.
on_chain_error(error, *, run_id[, parent_run_id])
Run when chain errors.
on_chain_start(serialized, inputs, **kwargs)
Print out that we are entering a chain.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id[, parent_run_id])
Run when LLM ends running.
on_llm_error(error, *, run_id[, parent_run_id])
Run when LLM errors.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Run when LLM starts running.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.
on_text(text[, color, end])
Run when agent ends.


on_text(text[, color, end])
Run when agent ends.
on_tool_end(output[, color, ...])
If not the final action, print out observation.
on_tool_error(error, *, run_id[, parent_run_id])
Run when tool errors.
on_tool_start(serialized, input_str, *, run_id)
Run when tool starts running.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, color: Optional[str] = None, **kwargs: Any) → Any[source]¶
Run on agent action.
on_agent_finish(finish: AgentFinish, color: Optional[str] = None, **kwargs: Any) → None[source]¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Print out that we finished a chain.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Print out that we are entering a chain.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.


Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM errors.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when LLM starts running.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.
on_text(text: str, color: Optional[str] = None, end: str = '', **kwargs: Any) → None[source]¶
Run when agent ends.


Run when agent ends.
on_tool_end(output: str, color: Optional[str] = None, observation_prefix: Optional[str] = None, llm_prefix: Optional[str] = None, **kwargs: Any) → None[source]¶
If not the final action, print out observation.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when tool starts running.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.tracers.stdout.try_json_stringify¶
langchain.callbacks.tracers.stdout.try_json_stringify(obj: Any, fallback: str) → str[source]¶
Try to stringify an object to JSON.
:param obj: Object to stringify.
:param fallback: Fallback string to return if the object cannot be stringified.
Returns
A JSON string if the object can be stringified, otherwise the fallback string.


langchain.callbacks.manager.tracing_v2_enabled¶
langchain.callbacks.manager.tracing_v2_enabled(project_name: Optional[str] = None, *, example_id: Optional[Union[UUID, str]] = None) → Generator[None, None, None][source]¶
Instruct LangChain to log all runs in context to LangSmith.
Parameters
project_name (str, optional) – The name of the project.
Defaults to “default”.
example_id (str or UUID, optional) – The ID of the example.
Defaults to None.
Returns
None
Example
>>> with tracing_v2_enabled():
...     # LangChain code will automatically be traced


langchain.callbacks.tracers.base.BaseTracer¶
class langchain.callbacks.tracers.base.BaseTracer(**kwargs: Any)[source]¶
Bases: BaseCallbackHandler, ABC
Base interface for tracers.
Methods
__init__(**kwargs)
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id, **kwargs)
End a trace for a chain run.
on_chain_error(error, *, run_id, **kwargs)
Handle an error for a chain run.
on_chain_start(serialized, inputs, *, run_id)
Start a trace for a chain run.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id, **kwargs)
End a trace for an LLM run.
on_llm_error(error, *, run_id, **kwargs)
Handle an error for an LLM run.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Start a trace for an LLM run.
on_retriever_end(documents, *, run_id, **kwargs)
Run when Retriever ends running.
on_retriever_error(error, *, run_id, **kwargs)
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])


on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id, **kwargs)
End a trace for a tool run.
on_tool_error(error, *, run_id, **kwargs)
Handle an error for a tool run.
on_tool_start(serialized, input_str, *, run_id)
Start a trace for a tool run.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, **kwargs: Any) → None[source]¶
End a trace for a chain run.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None[source]¶
Handle an error for a chain run.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None[source]¶
Start a trace for a chain run.


Start a trace for a chain run.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, **kwargs: Any) → None[source]¶
End a trace for an LLM run.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None[source]¶
Handle an error for an LLM run.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None[source]¶
Start a trace for an LLM run.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, **kwargs: Any) → None[source]¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None[source]¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None[source]¶
Run when Retriever starts running.


Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, **kwargs: Any) → None[source]¶
End a trace for a tool run.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None[source]¶
Handle an error for a tool run.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None[source]¶
Start a trace for a tool run.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.tracers.wandb.WandbTracer¶
class langchain.callbacks.tracers.wandb.WandbTracer(run_args: Optional[WandbRunArgs] = None, **kwargs: Any)[source]¶
Bases: BaseTracer
Callback Handler that logs to Weights and Biases.
This handler will log the model architecture and run traces to Weights and Biases.
This will ensure that all LangChain activity is logged to W&B.
Initializes the WandbTracer.
Parameters
run_args – (dict, optional) Arguments to pass to wandb.init(). If not
provided, wandb.init() will be called with no arguments. Please
refer to the wandb.init for more details.
To use W&B to monitor all LangChain activity, add this tracer like any other
LangChain callback:
```
from wandb.integration.langchain import WandbTracer
tracer = WandbTracer()
chain = LLMChain(llm, callbacks=[tracer])
# …end of notebook / script:
tracer.finish()
```
Methods
__init__([run_args])
Initializes the WandbTracer.
finish()
Waits for all asynchronous processes to finish and data to upload.
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id, **kwargs)
End a trace for a chain run.
on_chain_error(error, *, run_id, **kwargs)
Handle an error for a chain run.
on_chain_start(serialized, inputs, *, run_id)
Start a trace for a chain run.
on_chat_model_start(serialized, messages, *, ...)


on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id, **kwargs)
End a trace for an LLM run.
on_llm_error(error, *, run_id, **kwargs)
Handle an error for an LLM run.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Start a trace for an LLM run.
on_retriever_end(documents, *, run_id, **kwargs)
Run when Retriever ends running.
on_retriever_error(error, *, run_id, **kwargs)
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id, **kwargs)
End a trace for a tool run.
on_tool_error(error, *, run_id, **kwargs)
Handle an error for a tool run.
on_tool_start(serialized, input_str, *, run_id)
Start a trace for a tool run.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
finish() → None[source]¶
Waits for all asynchronous processes to finish and data to upload.


Waits for all asynchronous processes to finish and data to upload.
Proxy for wandb.finish().
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a chain run.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a chain run.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for a chain run.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for an LLM run.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for an LLM run.


Handle an error for an LLM run.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Start a trace for an LLM run.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, **kwargs: Any) → None¶
End a trace for a tool run.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, **kwargs: Any) → None¶
Handle an error for a tool run.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, tags: Optional[List[str]] = None, parent_run_id: Optional[UUID] = None, **kwargs: Any) → None¶


Start a trace for a tool run.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶
run_map: Dict[str, Run]¶


langchain.callbacks.tracers.schemas.LLMRun¶
class langchain.callbacks.tracers.schemas.LLMRun(*, uuid: str, parent_uuid: Optional[str] = None, start_time: datetime = None, end_time: datetime = None, extra: Optional[Dict[str, Any]] = None, execution_order: int, child_execution_order: int, serialized: Dict[str, Any], session_id: int, error: Optional[str] = None, prompts: List[str], response: Optional[LLMResult] = None)[source]¶
Bases: BaseRun
Class for LLMRun.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param child_execution_order: int [Required]¶
param end_time: datetime.datetime [Optional]¶
param error: Optional[str] = None¶
param execution_order: int [Required]¶
param extra: Optional[Dict[str, Any]] = None¶
param parent_uuid: Optional[str] = None¶
param prompts: List[str] [Required]¶
param response: Optional[langchain.schema.LLMResult] = None¶
param serialized: Dict[str, Any] [Required]¶
param session_id: int [Required]¶
param start_time: datetime.datetime [Optional]¶
param uuid: str [Required]¶


langchain.callbacks.human.HumanApprovalCallbackHandler¶
class langchain.callbacks.human.HumanApprovalCallbackHandler(approve: ~typing.Callable[[~typing.Any], bool] = <function _default_approve>, should_check: ~typing.Callable[[~typing.Dict[str, ~typing.Any]], bool] = <function _default_true>)[source]¶
Bases: BaseCallbackHandler
Callback for manually validating values.
Methods
__init__([approve, should_check])
on_agent_action(action, *, run_id[, ...])
Run on agent action.
on_agent_finish(finish, *, run_id[, ...])
Run on agent end.
on_chain_end(outputs, *, run_id[, parent_run_id])
Run when chain ends running.
on_chain_error(error, *, run_id[, parent_run_id])
Run when chain errors.
on_chain_start(serialized, inputs, *, run_id)
Run when chain starts running.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, *, run_id[, parent_run_id])
Run when LLM ends running.
on_llm_error(error, *, run_id[, parent_run_id])
Run when LLM errors.
on_llm_new_token(token, *, run_id[, ...])
Run on new LLM token.
on_llm_start(serialized, prompts, *, run_id)
Run when LLM starts running.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.


Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.
on_text(text, *, run_id[, parent_run_id])
Run on arbitrary text.
on_tool_end(output, *, run_id[, parent_run_id])
Run when tool ends running.
on_tool_error(error, *, run_id[, parent_run_id])
Run when tool errors.
on_tool_start(serialized, input_str, *, run_id)
Run when tool starts running.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
on_agent_action(action: AgentAction, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent action.
on_agent_finish(finish: AgentFinish, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when chain ends running.
on_chain_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when chain errors.


Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when chain starts running.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when LLM errors.
on_llm_new_token(token: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on new LLM token. Only available when streaming is enabled.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when LLM starts running.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.


Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.
on_text(text: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run on arbitrary text.
on_tool_end(output: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when tool ends running.
on_tool_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any[source]¶
Run when tool starts running.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = True¶
run_inline: bool = False¶


langchain.callbacks.tracers.schemas.Run¶
class langchain.callbacks.tracers.schemas.Run(*, id: UUID, name: str, start_time: datetime, run_type: RunTypeEnum, end_time: Optional[datetime] = None, extra: Optional[dict] = None, error: Optional[str] = None, serialized: Optional[dict] = None, events: Optional[List[Dict]] = None, inputs: dict, outputs: Optional[dict] = None, reference_example_id: Optional[UUID] = None, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, execution_order: int, child_execution_order: int, child_runs: List[Run] = None)[source]¶
Bases: RunBase
Run schema for the V2 API in the Tracer.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param child_execution_order: int [Required]¶
param child_runs: List[langchain.callbacks.tracers.schemas.Run] [Optional]¶
param end_time: Optional[<module 'datetime' from '/home/docs/.asdf/installs/python/3.11.4/lib/python3.11/datetime.py'>] = None¶
param error: Optional[str] = None¶
param events: Optional[List[Dict]] = None¶
param execution_order: int [Required]¶
param extra: Optional[dict] = None¶
param id: uuid.UUID [Required]¶
param inputs: dict [Required]¶
param name: str [Required]¶
param outputs: Optional[dict] = None¶
param parent_run_id: Optional[uuid.UUID] = None¶
param reference_example_id: Optional[uuid.UUID] = None¶


param reference_example_id: Optional[uuid.UUID] = None¶
param run_type: langchainplus_sdk.schemas.RunTypeEnum [Required]¶
param serialized: Optional[dict] = None¶
param start_time: <module 'datetime' from '/home/docs/.asdf/installs/python/3.11.4/lib/python3.11/datetime.py'> [Required]¶
param tags: Optional[List[str]] [Optional]¶
validator assign_name  »  all fields[source]¶
Assign name to the run.


langchain.callbacks.manager.AsyncCallbackManagerForToolRun¶
class langchain.callbacks.manager.AsyncCallbackManagerForToolRun(*, run_id: UUID, handlers: List[BaseCallbackHandler], inheritable_handlers: List[BaseCallbackHandler], parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, inheritable_tags: Optional[List[str]] = None)[source]¶
Bases: AsyncRunManager, ToolManagerMixin
Async callback manager for tool run.
Initialize the run manager.
Parameters
run_id (UUID) – The ID of the run.
handlers (List[BaseCallbackHandler]) – The list of handlers.
inheritable_handlers (List[BaseCallbackHandler]) – The list of inheritable handlers.
parent_run_id (UUID, optional) – The ID of the parent run.
Defaults to None.
tags (Optional[List[str]]) – The list of tags.
inheritable_tags (Optional[List[str]]) – The list of inheritable tags.
Methods
__init__(*, run_id, handlers, ...[, ...])
Initialize the run manager.
get_child([tag])
Get a child callback manager.
get_noop_manager()
Return a manager that doesn't perform any operations.
on_text(text, **kwargs)
Run when text is received.
on_tool_end(output, **kwargs)
Run when tool ends running.
on_tool_error(error, **kwargs)
Run when tool errors.
get_child(tag: Optional[str] = None) → AsyncCallbackManager[source]¶
Get a child callback manager.
Parameters
tag (str, optional) – The tag to add to the child
callback manager. Defaults to None.
Returns
The child callback manager.
Return type
AsyncCallbackManager
classmethod get_noop_manager() → BRM¶
Return a manager that doesn’t perform any operations.


Return a manager that doesn’t perform any operations.
Returns
The noop manager.
Return type
BaseRunManager
async on_text(text: str, **kwargs: Any) → Any¶
Run when text is received.
Parameters
text (str) – The received text.
Returns
The result of the callback.
Return type
Any
async on_tool_end(output: str, **kwargs: Any) → None[source]¶
Run when tool ends running.
Parameters
output (str) – The output of the tool.
async on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when tool errors.
Parameters
error (Exception or KeyboardInterrupt) – The error.


langchain.callbacks.tracers.stdout.elapsed¶
langchain.callbacks.tracers.stdout.elapsed(run: Any) → str[source]¶
Get the elapsed time of a run.
Parameters
run – any object with a start_time and end_time attribute.
Returns
A string with the elapsed time in seconds ormilliseconds if time is less than a second.


langchain.callbacks.openai_info.get_openai_token_cost_for_model¶
langchain.callbacks.openai_info.get_openai_token_cost_for_model(model_name: str, num_tokens: int, is_completion: bool = False) → float[source]¶
Get the cost in USD for a given model and number of tokens.
Parameters
model_name – Name of the model
num_tokens – Number of tokens.
is_completion – Whether the model is used for completion or not.
Defaults to False.
Returns
Cost in USD.


langchain.callbacks.whylabs_callback.WhyLabsCallbackHandler¶
class langchain.callbacks.whylabs_callback.WhyLabsCallbackHandler(logger: Logger)[source]¶
Bases: BaseCallbackHandler
WhyLabs CallbackHandler.
Initiate the rolling logger
Methods
__init__(logger)
Initiate the rolling logger
close()
flush()
from_params(*[, api_key, org_id, ...])
Instantiate whylogs Logger from params.
on_agent_action(action[, color])
Do nothing.
on_agent_finish(finish[, color])
Run on agent end.
on_chain_end(outputs, **kwargs)
Do nothing.
on_chain_error(error, **kwargs)
Do nothing.
on_chain_start(serialized, inputs, **kwargs)
Do nothing.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, **kwargs)
Pass the generated response to the logger.
on_llm_error(error, **kwargs)
Do nothing.
on_llm_new_token(token, **kwargs)
Do nothing.
on_llm_start(serialized, prompts, **kwargs)
Pass the input prompts to the logger
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.
on_text(text, **kwargs)
Do nothing.
on_tool_end(output[, color, ...])
Do nothing.
on_tool_error(error, **kwargs)
Do nothing.


Do nothing.
on_tool_error(error, **kwargs)
Do nothing.
on_tool_start(serialized, input_str, **kwargs)
Do nothing.
Attributes
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
close() → None[source]¶
flush() → None[source]¶
classmethod from_params(*, api_key: Optional[str] = None, org_id: Optional[str] = None, dataset_id: Optional[str] = None, sentiment: bool = False, toxicity: bool = False, themes: bool = False) → Logger[source]¶
Instantiate whylogs Logger from params.
Parameters
api_key (Optional[str]) – WhyLabs API key. Optional because the preferred
way to specify the API key is with environment variable
WHYLABS_API_KEY.
org_id (Optional[str]) – WhyLabs organization id to write profiles to.
If not set must be specified in environment variable
WHYLABS_DEFAULT_ORG_ID.
dataset_id (Optional[str]) – The model or dataset this callback is gathering
telemetry for. If not set must be specified in environment variable
WHYLABS_DEFAULT_DATASET_ID.
sentiment (bool) – If True will initialize a model to perform
sentiment analysis compound score. Defaults to False and will not gather
this metric.
toxicity (bool) – If True will initialize a model to score
toxicity. Defaults to False and will not gather this metric.
themes (bool) – If True will initialize a model to calculate
distance to configured themes. Defaults to None and will not gather this
metric.


distance to configured themes. Defaults to None and will not gather this
metric.
on_agent_action(action: AgentAction, color: Optional[str] = None, **kwargs: Any) → Any[source]¶
Do nothing.
on_agent_finish(finish: AgentFinish, color: Optional[str] = None, **kwargs: Any) → None[source]¶
Run on agent end.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Do nothing.
on_chain_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Do nothing.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Do nothing.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶
Run when a chat model starts running.
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Pass the generated response to the logger.
on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Do nothing.
on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Do nothing.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]¶
Pass the input prompts to the logger
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.


Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.
on_text(text: str, **kwargs: Any) → None[source]¶
Do nothing.
on_tool_end(output: str, color: Optional[str] = None, observation_prefix: Optional[str] = None, llm_prefix: Optional[str] = None, **kwargs: Any) → None[source]¶
Do nothing.
on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Do nothing.
on_tool_start(serialized: Dict[str, Any], input_str: str, **kwargs: Any) → None[source]¶
Do nothing.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.wandb_callback.import_wandb¶
langchain.callbacks.wandb_callback.import_wandb() → Any[source]¶
Import the wandb python package and raise an error if it is not installed.


langchain.callbacks.comet_ml_callback.CometCallbackHandler¶
class langchain.callbacks.comet_ml_callback.CometCallbackHandler(task_type: Optional[str] = 'inference', workspace: Optional[str] = None, project_name: Optional[str] = None, tags: Optional[Sequence] = None, name: Optional[str] = None, visualizations: Optional[List[str]] = None, complexity_metrics: bool = False, custom_metrics: Optional[Callable] = None, stream_logs: bool = True)[source]¶
Bases: BaseMetadataCallbackHandler, BaseCallbackHandler
Callback Handler that logs to Comet.
Parameters
job_type (str) – The type of comet_ml task such as “inference”,
“testing” or “qc”
project_name (str) – The comet_ml project name
tags (list) – Tags to add to the task
task_name (str) – Name of the comet_ml task
visualize (bool) – Whether to visualize the run.
complexity_metrics (bool) – Whether to log complexity metrics
stream_logs (bool) – Whether to stream callback actions to Comet
This handler will utilize the associated callback method and formats
the input of each callback function with metadata regarding the state of LLM run,
and adds the response to the list of records for both the {method}_records and
action. It then logs the response to Comet.
Initialize callback handler.
Methods
__init__([task_type, workspace, ...])
Initialize callback handler.
flush_tracker([langchain_asset, task_type, ...])
Flush the tracker and setup the session.
get_custom_callback_meta()
on_agent_action(action, **kwargs)
Run on agent action.
on_agent_finish(finish, **kwargs)
Run when agent ends running.
on_chain_end(outputs, **kwargs)
Run when chain ends running.


on_chain_end(outputs, **kwargs)
Run when chain ends running.
on_chain_error(error, **kwargs)
Run when chain errors.
on_chain_start(serialized, inputs, **kwargs)
Run when chain starts running.
on_chat_model_start(serialized, messages, *, ...)
Run when a chat model starts running.
on_llm_end(response, **kwargs)
Run when LLM ends running.
on_llm_error(error, **kwargs)
Run when LLM errors.
on_llm_new_token(token, **kwargs)
Run when LLM generates a new token.
on_llm_start(serialized, prompts, **kwargs)
Run when LLM starts.
on_retriever_end(documents, *, run_id[, ...])
Run when Retriever ends running.
on_retriever_error(error, *, run_id[, ...])
Run when Retriever errors.
on_retriever_start(query, *, run_id[, ...])
Run when Retriever starts running.
on_text(text, **kwargs)
Run when agent is ending.
on_tool_end(output, **kwargs)
Run when tool ends running.
on_tool_error(error, **kwargs)
Run when tool errors.
on_tool_start(serialized, input_str, **kwargs)
Run when tool starts running.
reset_callback_meta()
Reset the callback metadata.
Attributes
always_verbose
Whether to call verbose callbacks even if verbose is False.
ignore_agent
Whether to ignore agent callbacks.
ignore_chain
Whether to ignore chain callbacks.
ignore_chat_model
Whether to ignore chat model callbacks.
ignore_llm
Whether to ignore LLM callbacks.
ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline


ignore_retriever
Whether to ignore retriever callbacks.
raise_error
run_inline
flush_tracker(langchain_asset: Any = None, task_type: Optional[str] = 'inference', workspace: Optional[str] = None, project_name: Optional[str] = 'comet-langchain-demo', tags: Optional[Sequence] = None, name: Optional[str] = None, visualizations: Optional[List[str]] = None, complexity_metrics: bool = False, custom_metrics: Optional[Callable] = None, finish: bool = False, reset: bool = False) → None[source]¶
Flush the tracker and setup the session.
Everything after this will be a new table.
Parameters
name – Name of the preformed session so far so it is identifyable
langchain_asset – The langchain asset to save.
finish – Whether to finish the run.
Returns – None
get_custom_callback_meta() → Dict[str, Any]¶
on_agent_action(action: AgentAction, **kwargs: Any) → Any[source]¶
Run on agent action.
on_agent_finish(finish: AgentFinish, **kwargs: Any) → None[source]¶
Run when agent ends running.
on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain ends running.
on_chain_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when chain errors.
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]¶
Run when chain starts running.
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], *, run_id: UUID, parent_run_id: Optional[UUID] = None, tags: Optional[List[str]] = None, **kwargs: Any) → Any¶


Run when a chat model starts running.
on_llm_end(response: LLMResult, **kwargs: Any) → None[source]¶
Run when LLM ends running.
on_llm_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when LLM errors.
on_llm_new_token(token: str, **kwargs: Any) → None[source]¶
Run when LLM generates a new token.
on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]¶
Run when LLM starts.
on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever ends running.
on_retriever_error(error: Union[Exception, KeyboardInterrupt], *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever errors.
on_retriever_start(query: str, *, run_id: UUID, parent_run_id: Optional[UUID] = None, **kwargs: Any) → Any¶
Run when Retriever starts running.
on_text(text: str, **kwargs: Any) → None[source]¶
Run when agent is ending.
on_tool_end(output: str, **kwargs: Any) → None[source]¶
Run when tool ends running.
on_tool_error(error: Union[Exception, KeyboardInterrupt], **kwargs: Any) → None[source]¶
Run when tool errors.
on_tool_start(serialized: Dict[str, Any], input_str: str, **kwargs: Any) → None[source]¶
Run when tool starts running.
reset_callback_meta() → None¶
Reset the callback metadata.


reset_callback_meta() → None¶
Reset the callback metadata.
property always_verbose: bool¶
Whether to call verbose callbacks even if verbose is False.
property ignore_agent: bool¶
Whether to ignore agent callbacks.
property ignore_chain: bool¶
Whether to ignore chain callbacks.
property ignore_chat_model: bool¶
Whether to ignore chat model callbacks.
property ignore_llm: bool¶
Whether to ignore LLM callbacks.
property ignore_retriever: bool¶
Whether to ignore retriever callbacks.
raise_error: bool = False¶
run_inline: bool = False¶


langchain.callbacks.manager.CallbackManager¶
class langchain.callbacks.manager.CallbackManager(handlers: List[BaseCallbackHandler], inheritable_handlers: Optional[List[BaseCallbackHandler]] = None, parent_run_id: Optional[UUID] = None, *, tags: Optional[List[str]] = None, inheritable_tags: Optional[List[str]] = None)[source]¶
Bases: BaseCallbackManager
Callback manager that can be used to handle callbacks from langchain.
Initialize callback manager.
Methods
__init__(handlers[, inheritable_handlers, ...])
Initialize callback manager.
add_handler(handler[, inherit])
Add a handler to the callback manager.
add_tags(tags[, inherit])
configure([inheritable_callbacks, ...])
Configure the callback manager.
on_chain_start(serialized, inputs[, run_id])
Run when chain starts running.
on_chat_model_start(serialized, messages, ...)
Run when LLM starts running.
on_llm_start(serialized, prompts, **kwargs)
Run when LLM starts running.
on_retriever_start(query[, run_id, ...])
Run when retriever starts running.
on_tool_start(serialized, input_str[, ...])
Run when tool starts running.
remove_handler(handler)
Remove a handler from the callback manager.
remove_tags(tags)
set_handler(handler[, inherit])
Set handler as the only handler on the callback manager.
set_handlers(handlers[, inherit])
Set handlers as the only handlers on the callback manager.
Attributes
is_async
Whether the callback manager is async.
add_handler(handler: BaseCallbackHandler, inherit: bool = True) → None¶
Add a handler to the callback manager.
add_tags(tags: List[str], inherit: bool = True) → None¶


add_tags(tags: List[str], inherit: bool = True) → None¶
classmethod configure(inheritable_callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, local_callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, verbose: bool = False, inheritable_tags: Optional[List[str]] = None, local_tags: Optional[List[str]] = None) → CallbackManager[source]¶
Configure the callback manager.
Parameters
inheritable_callbacks (Optional[Callbacks], optional) – The inheritable
callbacks. Defaults to None.
local_callbacks (Optional[Callbacks], optional) – The local callbacks.
Defaults to None.
verbose (bool, optional) – Whether to enable verbose mode. Defaults to False.
inheritable_tags (Optional[List[str]], optional) – The inheritable tags.
Defaults to None.
local_tags (Optional[List[str]], optional) – The local tags.
Defaults to None.
Returns
The configured callback manager.
Return type
CallbackManager
on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], run_id: Optional[UUID] = None, **kwargs: Any) → CallbackManagerForChainRun[source]¶
Run when chain starts running.
Parameters
serialized (Dict[str, Any]) – The serialized chain.
inputs (Dict[str, Any]) – The inputs to the chain.
run_id (UUID, optional) – The ID of the run. Defaults to None.
Returns
The callback manager for the chain run.
Return type
CallbackManagerForChainRun
on_chat_model_start(serialized: Dict[str, Any], messages: List[List[BaseMessage]], **kwargs: Any) → List[CallbackManagerForLLMRun][source]¶
Run when LLM starts running.
Parameters
serialized (Dict[str, Any]) – The serialized LLM.