如何修剪消息

先决条件

本指南假定您熟悉以下概念

本指南中的方法也需要 langchain-core>=0.2.9。

所有模型都有有限的上下文窗口，这意味着它们可以接受的令牌数量有限。如果您有非常长的消息或累积了大量消息历史记录的链/代理，则需要管理传递给模型的消息长度。

trim_messages 可用于将聊天历史记录的大小减少到指定的令牌计数或指定的邮件计数。

如果将修剪后的聊天历史记录直接传递回聊天模型，则修剪后的聊天历史记录应满足以下属性

生成的聊天历史记录应为有效的。通常这意味着应满足以下属性
- 聊天历史记录以 (1) HumanMessage 或 (2) SystemMessage 后跟 HumanMessage 开始。
- 聊天历史记录以 HumanMessage 或 ToolMessage 结尾。
- ToolMessage 只能出现在涉及工具调用的 AIMessage 之后。这可以通过设置 start_on="human" 和 ends_on=("human", "tool") 来实现。
它包含最近的消息，并在聊天历史记录中删除旧消息。这可以通过设置 strategy="last" 来实现。
通常，新的聊天历史记录应包含 SystemMessage（如果它存在于原始聊天历史记录中），因为 SystemMessage 包含发送给聊天模型的特殊指令。SystemMessage 通常是历史记录中（如果存在）的第一条消息。这可以通过设置 include_system=True 来实现。

基于令牌计数的修剪

在这里，我们将根据令牌计数修剪聊天历史记录。修剪后的聊天历史记录将生成一个包含 SystemMessage 的有效聊天历史记录。

为了保留最新的消息，我们将设置 strategy="last"。我们还将设置 include_system=True 以包含 SystemMessage，并将 start_on="human" 设置为确保生成的聊天历史记录有效。

当基于令牌计数使用 trim_messages 时，这是一个很好的默认配置。请记住根据您的用例调整 token_counter 和 max_tokens。

请注意，对于我们的 token_counter，我们可以传入一个函数（稍后将详细介绍）或一个语言模型（因为语言模型具有消息令牌计数方法）。当你需要调整消息以适应特定模型的上下文窗口时，传入模型是有意义的。

pip install -qU langchain-openai

Note: you may need to restart the kernel to use updated packages.

from langchain_core.messages import (
    AIMessage,
    HumanMessage,
    SystemMessage,
    ToolMessage,
    trim_messages,
)
from langchain_openai import ChatOpenAI

messages = [
    SystemMessage("you're a good assistant, you always respond with a joke."),
    HumanMessage("i wonder why it's called langchain"),
    AIMessage(
        'Well, I guess they thought "WordRope" and "SentenceString" just didn\'t have the same ring to it!'
    ),
    HumanMessage("and who is harrison chasing anyways"),
    AIMessage(
        "Hmmm let me think.\n\nWhy, he's probably chasing after the last cup of coffee in the office!"
    ),
    HumanMessage("what do you call a speechless parrot"),
]


trim_messages(
    messages,
    # Keep the last <= n_count tokens of the messages.
    strategy="last",
    # Remember to adjust based on your model
    # or else pass a custom token_encoder
    token_counter=ChatOpenAI(model="gpt-4o"),
    # Most chat models expect that chat history starts with either:
    # (1) a HumanMessage or
    # (2) a SystemMessage followed by a HumanMessage
    # Remember to adjust based on the desired conversation
    # length
    max_tokens=45,
    # Most chat models expect that chat history starts with either:
    # (1) a HumanMessage or
    # (2) a SystemMessage followed by a HumanMessage
    start_on="human",
    # Most chat models expect that chat history ends with either:
    # (1) a HumanMessage or
    # (2) a ToolMessage
    end_on=("human", "tool"),
    # Usually, we want to keep the SystemMessage
    # if it's present in the original history.
    # The SystemMessage has special instructions for the model.
    include_system=True,
    allow_partial=False,
)

[SystemMessage(content="you're a good assistant, you always respond with a joke.", additional_kwargs={}, response_metadata={}),
 HumanMessage(content='what do you call a speechless parrot', additional_kwargs={}, response_metadata={})]

基于消息数量的修剪

或者，我们可以通过设置 token_counter=len，基于 **消息数量** 来修剪聊天记录。在这种情况下，每条消息将被视为一个令牌，而 max_tokens 将控制消息的最大数量。

当基于消息数量使用 trim_messages 时，这是一个很好的默认配置。请记住根据你的用例调整 max_tokens。

trim_messages(
    messages,
    # Keep the last <= n_count tokens of the messages.
    strategy="last",
    token_counter=len,
    # When token_counter=len, each message
    # will be counted as a single token.
    # Remember to adjust for your use case
    max_tokens=5,
    # Most chat models expect that chat history starts with either:
    # (1) a HumanMessage or
    # (2) a SystemMessage followed by a HumanMessage
    start_on="human",
    # Most chat models expect that chat history ends with either:
    # (1) a HumanMessage or
    # (2) a ToolMessage
    end_on=("human", "tool"),
    # Usually, we want to keep the SystemMessage
    # if it's present in the original history.
    # The SystemMessage has special instructions for the model.
    include_system=True,
)

[SystemMessage(content="you're a good assistant, you always respond with a joke.", additional_kwargs={}, response_metadata={}),
 HumanMessage(content='and who is harrison chasing anyways', additional_kwargs={}, response_metadata={}),
 AIMessage(content="Hmmm let me think.\n\nWhy, he's probably chasing after the last cup of coffee in the office!", additional_kwargs={}, response_metadata={}),
 HumanMessage(content='what do you call a speechless parrot', additional_kwargs={}, response_metadata={})]

高级用法

你可以使用 trim_message 作为构建块来创建更复杂的处理逻辑。

如果我们想要允许拆分消息内容，我们可以指定 allow_partial=True

trim_messages(
    messages,
    max_tokens=56,
    strategy="last",
    token_counter=ChatOpenAI(model="gpt-4o"),
    include_system=True,
    allow_partial=True,
)

[SystemMessage(content="you're a good assistant, you always respond with a joke.", additional_kwargs={}, response_metadata={}),
 AIMessage(content="\nWhy, he's probably chasing after the last cup of coffee in the office!", additional_kwargs={}, response_metadata={}),
 HumanMessage(content='what do you call a speechless parrot', additional_kwargs={}, response_metadata={})]

默认情况下，SystemMessage 将不会被包含在内，因此你可以通过设置 include_system=False 或删除 include_system 参数来删除它。

trim_messages(
    messages,
    max_tokens=45,
    strategy="last",
    token_counter=ChatOpenAI(model="gpt-4o"),
)

[AIMessage(content="Hmmm let me think.\n\nWhy, he's probably chasing after the last cup of coffee in the office!", additional_kwargs={}, response_metadata={}),
 HumanMessage(content='what do you call a speechless parrot', additional_kwargs={}, response_metadata={})]

我们可以通过指定 strategy="first" 来执行获取前 max_tokens 的反向操作。

trim_messages(
    messages,
    max_tokens=45,
    strategy="first",
    token_counter=ChatOpenAI(model="gpt-4o"),
)

[SystemMessage(content="you're a good assistant, you always respond with a joke.", additional_kwargs={}, response_metadata={}),
 HumanMessage(content="i wonder why it's called langchain", additional_kwargs={}, response_metadata={})]

编写自定义令牌计数器

我们可以编写一个自定义令牌计数器函数，该函数接收一个消息列表并返回一个整数。

pip install -qU tiktoken

Note: you may need to restart the kernel to use updated packages.

from typing import List

import tiktoken
from langchain_core.messages import BaseMessage, ToolMessage


def str_token_counter(text: str) -> int:
    enc = tiktoken.get_encoding("o200k_base")
    return len(enc.encode(text))


def tiktoken_counter(messages: List[BaseMessage]) -> int:
    """Approximately reproduce https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb

    For simplicity only supports str Message.contents.
    """
    num_tokens = 3  # every reply is primed with <|start|>assistant<|message|>
    tokens_per_message = 3
    tokens_per_name = 1
    for msg in messages:
        if isinstance(msg, HumanMessage):
            role = "user"
        elif isinstance(msg, AIMessage):
            role = "assistant"
        elif isinstance(msg, ToolMessage):
            role = "tool"
        elif isinstance(msg, SystemMessage):
            role = "system"
        else:
            raise ValueError(f"Unsupported messages type {msg.__class__}")
        num_tokens += (
            tokens_per_message
            + str_token_counter(role)
            + str_token_counter(msg.content)
        )
        if msg.name:
            num_tokens += tokens_per_name + str_token_counter(msg.name)
    return num_tokens


trim_messages(
    messages,
    token_counter=tiktoken_counter,
    # Keep the last <= n_count tokens of the messages.
    strategy="last",
    # When token_counter=len, each message
    # will be counted as a single token.
    # Remember to adjust for your use case
    max_tokens=45,
    # Most chat models expect that chat history starts with either:
    # (1) a HumanMessage or
    # (2) a SystemMessage followed by a HumanMessage
    start_on="human",
    # Most chat models expect that chat history ends with either:
    # (1) a HumanMessage or
    # (2) a ToolMessage
    end_on=("human", "tool"),
    # Usually, we want to keep the SystemMessage
    # if it's present in the original history.
    # The SystemMessage has special instructions for the model.
    include_system=True,
)

API 参考：BaseMessage | ToolMessage

[SystemMessage(content="you're a good assistant, you always respond with a joke.", additional_kwargs={}, response_metadata={}),
 HumanMessage(content='what do you call a speechless parrot', additional_kwargs={}, response_metadata={})]

链接

trim_messages 可以以命令式方式（如上所述）或声明式方式使用，使它易于与链中的其他组件组合。

llm = ChatOpenAI(model="gpt-4o")

# Notice we don't pass in messages. This creates
# a RunnableLambda that takes messages as input
trimmer = trim_messages(
    token_counter=llm,
    # Keep the last <= n_count tokens of the messages.
    strategy="last",
    # When token_counter=len, each message
    # will be counted as a single token.
    # Remember to adjust for your use case
    max_tokens=45,
    # Most chat models expect that chat history starts with either:
    # (1) a HumanMessage or
    # (2) a SystemMessage followed by a HumanMessage
    start_on="human",
    # Most chat models expect that chat history ends with either:
    # (1) a HumanMessage or
    # (2) a ToolMessage
    end_on=("human", "tool"),
    # Usually, we want to keep the SystemMessage
    # if it's present in the original history.
    # The SystemMessage has special instructions for the model.
    include_system=True,
)

chain = trimmer | llm
chain.invoke(messages)

AIMessage(content='A polygon! Because it\'s a "poly-gone" quiet!', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 13, 'prompt_tokens': 32, 'total_tokens': 45, 'completion_tokens_details': {'reasoning_tokens': 0}}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_057232b607', 'finish_reason': 'stop', 'logprobs': None}, id='run-4fa026e7-9137-4fef-b596-54243615e3b3-0', usage_metadata={'input_tokens': 32, 'output_tokens': 13, 'total_tokens': 45})

查看 LangSmith 跟踪，我们可以看到，在消息传递给模型之前，它们首先被修剪：https://smith.langchain.com/public/65af12c4-c24d-4824-90f0-6547566e59bb/r

仅查看修剪器，我们可以看到它是一个可运行的对象，可以像所有可运行对象一样被调用。

trimmer.invoke(messages)

[SystemMessage(content="you're a good assistant, you always respond with a joke.", additional_kwargs={}, response_metadata={}),
 HumanMessage(content='what do you call a speechless parrot', additional_kwargs={}, response_metadata={})]

与 ChatMessageHistory 一起使用

修剪消息在处理聊天记录时特别有用，因为聊天记录可以无限增长。

from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory

chat_history = InMemoryChatMessageHistory(messages=messages[:-1])


def dummy_get_session_history(session_id):
    if session_id != "1":
        return InMemoryChatMessageHistory()
    return chat_history


llm = ChatOpenAI(model="gpt-4o")

trimmer = trim_messages(
    max_tokens=45,
    strategy="last",
    token_counter=llm,
    # Usually, we want to keep the SystemMessage
    # if it's present in the original history.
    # The SystemMessage has special instructions for the model.
    include_system=True,
    # Most chat models expect that chat history starts with either:
    # (1) a HumanMessage or
    # (2) a SystemMessage followed by a HumanMessage
    # start_on="human" makes sure we produce a valid chat history
    start_on="human",
)

chain = trimmer | llm
chain_with_history = RunnableWithMessageHistory(chain, dummy_get_session_history)
chain_with_history.invoke(
    [HumanMessage("what do you call a speechless parrot")],
    config={"configurable": {"session_id": "1"}},
)

API 参考：InMemoryChatMessageHistory | RunnableWithMessageHistory

AIMessage(content='A "polygon"!', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 4, 'prompt_tokens': 32, 'total_tokens': 36, 'completion_tokens_details': {'reasoning_tokens': 0}}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_c17d3befe7', 'finish_reason': 'stop', 'logprobs': None}, id='run-71d9fce6-bb0c-4bb3-acc8-d5eaee6ae7bc-0', usage_metadata={'input_tokens': 32, 'output_tokens': 4, 'total_tokens': 36})

查看 LangSmith 跟踪，我们可以看到，我们检索了所有消息，但在将消息传递给模型之前，它们被修剪为仅包含系统消息和最后一条人类消息：https://smith.langchain.com/public/17dd700b-9994-44ca-930c-116e00997315/r

API 参考

有关所有参数的完整描述，请访问 API 参考：https://python.langchain.ac.cn/api_reference/core/messages/langchain_core.messages.utils.trim_messages.html

如何修剪消息

基于令牌计数的修剪

基于消息数量的修剪

高级用法

编写自定义令牌计数器

链接

与 ChatMessageHistory 一起使用

API 参考

此页面对您有帮助吗？

您也可以在 GitHub 上留下详细的反馈。 GitHub.

基于令牌计数的修剪​

基于消息数量的修剪​

高级用法​

编写自定义令牌计数器​

链接​

与 ChatMessageHistory 一起使用​

API 参考​

此页面对您有帮助吗？

您也可以在 GitHub 上留下详细的反馈。 GitHub.

基于令牌计数的修剪

基于消息数量的修剪

高级用法

编写自定义令牌计数器

链接

与 ChatMessageHistory 一起使用

API 参考