Give pipeline authors options for how history is managed. In the UI this should be presented as a dropdown + additional fields under the advanced editor. On the node itself, the configured mode should be displayed in small text below the history dropdown.

Modes

Summarize

If the token count is > X tokens, summarize the history but keep the last N messages.

Config options:

• X: Token Limit
  • help text: When the token count of the history + prompt + user input exceeds this number, the history will be compressed by summarizing the oldest messages.

Truncate Tokens

If the token count is > X tokens, truncate the history until it is < X tokens. This works by removing older messages until the token count meets the requirement.

Config options:

• X: Token Limit
  • help text: When the token count of the history + prompt + user input exceeds this number, the history will be truncated by removing the oldest messages until the token count is below the limit.

Max history length

Always truncate the history to the last N messages.

Config options:

• N: Max History Length
  • help text: The chat history that is sent to the LLM will be limited to this number of messages. Older messages will not be sent.