跳到主要内容

如何处理工具错误

前提条件

本指南假设您熟悉以下概念

使用 LLM 调用工具通常比纯粹的提示更可靠,但它并非完美。模型可能会尝试调用不存在的工具,或者未能返回与请求的模式匹配的参数。诸如保持模式简单、减少一次传递的工具数量以及使用良好的名称和描述等策略可以帮助缓解这种风险,但并非万无一失。

本指南介绍了一些在链中构建错误处理的方法,以减轻这些失败模式。

设置

我们需要安装以下软件包

%pip install --upgrade --quiet langchain-core langchain-openai

如果您想在 LangSmith 中跟踪您的运行,请取消注释并设置以下环境变量

import getpass
import os

# os.environ["LANGCHAIN_TRACING_V2"] = "true"
# os.environ["LANGCHAIN_API_KEY"] = getpass.getpass()

假设我们有以下(虚拟)工具和工具调用链。我们将故意使我们的工具变得复杂,以尝试绊倒模型。

pip install -qU langchain-openai
import getpass
import os

if not os.environ.get("OPENAI_API_KEY"):
os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter API key for OpenAI: ")

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")
# Define tool
from langchain_core.tools import tool


@tool
def complex_tool(int_arg: int, float_arg: float, dict_arg: dict) -> int:
"""Do something complex with a complex tool."""
return int_arg * float_arg


llm_with_tools = llm.bind_tools(
[complex_tool],
)

# Define chain
chain = llm_with_tools | (lambda msg: msg.tool_calls[0]["args"]) | complex_tool
API 参考:tool

我们可以看到,当我们尝试使用即使是相当明确的输入来调用此链时,模型也未能正确调用该工具(它忘记了 `dict_arg` 参数)。

chain.invoke(
"use complex tool. the args are 5, 2.1, empty dictionary. don't forget dict_arg"
)
---------------------------------------------------------------------------
``````output
ValidationError Traceback (most recent call last)
``````output
Cell In[5], line 1
----> 1 chain.invoke(
2 "use complex tool. the args are 5, 2.1, empty dictionary. don't forget dict_arg"
3 )
``````output
File ~/langchain/.venv/lib/python3.11/site-packages/langchain_core/runnables/base.py:2998, in RunnableSequence.invoke(self, input, config, **kwargs)
2996 input = context.run(step.invoke, input, config, **kwargs)
2997 else:
-> 2998 input = context.run(step.invoke, input, config)
2999 # finish the root run
3000 except BaseException as e:
``````output
File ~/langchain/.venv/lib/python3.11/site-packages/langchain_core/tools/base.py:456, in BaseTool.invoke(self, input, config, **kwargs)
449 def invoke(
450 self,
451 input: Union[str, Dict, ToolCall],
452 config: Optional[RunnableConfig] = None,
453 **kwargs: Any,
454 ) -> Any:
455 tool_input, kwargs = _prep_run_args(input, config, **kwargs)
--> 456 return self.run(tool_input, **kwargs)
``````output
File ~/langchain/.venv/lib/python3.11/site-packages/langchain_core/tools/base.py:659, in BaseTool.run(self, tool_input, verbose, start_color, color, callbacks, tags, metadata, run_name, run_id, config, tool_call_id, **kwargs)
657 if error_to_raise:
658 run_manager.on_tool_error(error_to_raise)
--> 659 raise error_to_raise
660 output = _format_output(content, artifact, tool_call_id, self.name, status)
661 run_manager.on_tool_end(output, color=color, name=self.name, **kwargs)
``````output
File ~/langchain/.venv/lib/python3.11/site-packages/langchain_core/tools/base.py:622, in BaseTool.run(self, tool_input, verbose, start_color, color, callbacks, tags, metadata, run_name, run_id, config, tool_call_id, **kwargs)
620 context = copy_context()
621 context.run(_set_config_context, child_config)
--> 622 tool_args, tool_kwargs = self._to_args_and_kwargs(tool_input)
623 if signature(self._run).parameters.get("run_manager"):
624 tool_kwargs["run_manager"] = run_manager
``````output
File ~/langchain/.venv/lib/python3.11/site-packages/langchain_core/tools/base.py:545, in BaseTool._to_args_and_kwargs(self, tool_input)
544 def _to_args_and_kwargs(self, tool_input: Union[str, Dict]) -> Tuple[Tuple, Dict]:
--> 545 tool_input = self._parse_input(tool_input)
546 # For backwards compatibility, if run_input is a string,
547 # pass as a positional argument.
548 if isinstance(tool_input, str):
``````output
File ~/langchain/.venv/lib/python3.11/site-packages/langchain_core/tools/base.py:487, in BaseTool._parse_input(self, tool_input)
485 if input_args is not None:
486 if issubclass(input_args, BaseModel):
--> 487 result = input_args.model_validate(tool_input)
488 result_dict = result.model_dump()
489 elif issubclass(input_args, BaseModelV1):
``````output
File ~/langchain/.venv/lib/python3.11/site-packages/pydantic/main.py:568, in BaseModel.model_validate(cls, obj, strict, from_attributes, context)
566 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
567 __tracebackhide__ = True
--> 568 return cls.__pydantic_validator__.validate_python(
569 obj, strict=strict, from_attributes=from_attributes, context=context
570 )
``````output
ValidationError: 1 validation error for complex_toolSchema
dict_arg
Field required [type=missing, input_value={'int_arg': 5, 'float_arg': 2.1}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.8/v/missing

Try/except 工具调用

更优雅地处理错误的最简单方法是尝试/捕获工具调用步骤并在错误时返回有用的消息

from typing import Any

from langchain_core.runnables import Runnable, RunnableConfig


def try_except_tool(tool_args: dict, config: RunnableConfig) -> Runnable:
try:
complex_tool.invoke(tool_args, config=config)
except Exception as e:
return f"Calling tool with arguments:\n\n{tool_args}\n\nraised the following error:\n\n{type(e)}: {e}"


chain = llm_with_tools | (lambda msg: msg.tool_calls[0]["args"]) | try_except_tool

print(
chain.invoke(
"use complex tool. the args are 5, 2.1, empty dictionary. don't forget dict_arg"
)
)
API 参考:Runnable | RunnableConfig
Calling tool with arguments:

{'int_arg': 5, 'float_arg': 2.1}

raised the following error:

<class 'pydantic_core._pydantic_core.ValidationError'>: 1 validation error for complex_toolSchema
dict_arg
Field required [type=missing, input_value={'int_arg': 5, 'float_arg': 2.1}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.8/v/missing

回退

我们还可以尝试在发生工具调用错误时回退到更好的模型。在这种情况下,我们将回退到使用 `gpt-4-1106-preview` 而不是 `gpt-3.5-turbo` 的相同链。

chain = llm_with_tools | (lambda msg: msg.tool_calls[0]["args"]) | complex_tool

better_model = ChatOpenAI(model="gpt-4-1106-preview", temperature=0).bind_tools(
[complex_tool], tool_choice="complex_tool"
)

better_chain = better_model | (lambda msg: msg.tool_calls[0]["args"]) | complex_tool

chain_with_fallback = chain.with_fallbacks([better_chain])

chain_with_fallback.invoke(
"use complex tool. the args are 5, 2.1, empty dictionary. don't forget dict_arg"
)
10.5

查看此链运行的 LangSmith 跟踪,我们可以看到第一个链调用如预期的那样失败,而回退成功了。

使用异常重试

为了更进一步,我们可以尝试使用传入的异常自动重新运行链,以便模型能够纠正其行为

from langchain_core.messages import AIMessage, HumanMessage, ToolCall, ToolMessage
from langchain_core.prompts import ChatPromptTemplate


class CustomToolException(Exception):
"""Custom LangChain tool exception."""

def __init__(self, tool_call: ToolCall, exception: Exception) -> None:
super().__init__()
self.tool_call = tool_call
self.exception = exception


def tool_custom_exception(msg: AIMessage, config: RunnableConfig) -> Runnable:
try:
return complex_tool.invoke(msg.tool_calls[0]["args"], config=config)
except Exception as e:
raise CustomToolException(msg.tool_calls[0], e)


def exception_to_messages(inputs: dict) -> dict:
exception = inputs.pop("exception")

# Add historical messages to the original input, so the model knows that it made a mistake with the last tool call.
messages = [
AIMessage(content="", tool_calls=[exception.tool_call]),
ToolMessage(
tool_call_id=exception.tool_call["id"], content=str(exception.exception)
),
HumanMessage(
content="The last tool call raised an exception. Try calling the tool again with corrected arguments. Do not repeat mistakes."
),
]
inputs["last_output"] = messages
return inputs


# We add a last_output MessagesPlaceholder to our prompt which if not passed in doesn't
# affect the prompt at all, but gives us the option to insert an arbitrary list of Messages
# into the prompt if needed. We'll use this on retries to insert the error message.
prompt = ChatPromptTemplate.from_messages(
[("human", "{input}"), ("placeholder", "{last_output}")]
)
chain = prompt | llm_with_tools | tool_custom_exception

# If the initial chain call fails, we rerun it withe the exception passed in as a message.
self_correcting_chain = chain.with_fallbacks(
[exception_to_messages | chain], exception_key="exception"
)
self_correcting_chain.invoke(
{
"input": "use complex tool. the args are 5, 2.1, empty dictionary. don't forget dict_arg"
}
)
10.5

我们的链成功了!查看 LangSmith 跟踪,我们可以看到我们初始的链确实仍然失败,只有在重试后链才成功。

下一步

现在您已经了解了一些处理工具调用错误的策略。接下来,您可以了解更多关于如何使用工具的信息

您还可以查看一些更具体的工具调用使用方法


此页是否有帮助?