Summary-based context compaction.
When the conversation approaches the configured reserve threshold, Alloy preserves the first message, keeps a recent verbatim token window, and replaces older context with a structured handoff summary. If summary generation fails, Alloy falls back to deterministic truncation.
Compaction never splits a tool round: a kept tool result always keeps the
assistant tool call before it, and truncation keeps the turn in progress
whole. Because compaction edits earlier history, it removes every
thinking and redacted_thinking block it keeps, including the turn in
progress: signed thinking is bound to the history it was produced after.
Under manual thinking ("type" => "enabled" in :extra_body), which
requires the turn in progress to start with thinking and whose models do
not check history, that turn keeps its thinking. If the compacted request
is still
over budget, the largest retained tool results are shortened, keeping
their beginning and a [tool result truncated: ...] marker.
Compaction options live under compaction::
:clear_tool_results- clear old bulky tool-result content before summary generation (defaulttrue):keep_recent_tool_results- newest tool results to preserve verbatim when clearing (default3):summary_system_prompt- system prompt used for summary generation:summary_prompt- user prompt instructions appended after the serialized conversation
Summary
Functions
Compaction as middleware.
Compacts messages, preserving the first message and the most recent N messages.
Forces compaction regardless of reserve budget. Used when the provider rejects the prompt as too long.
Compacts state messages when the estimated prompt exceeds
max_tokens - reserve_tokens.
Prefix used for synthetic handoff summary messages inserted by the compactor.
Functions
@spec call(Alloy.Middleware.hook(), Alloy.Agent.State.t()) :: Alloy.Agent.State.t() | {:halt, String.t(), Alloy.Agent.State.t()}
Compaction as middleware.
On :before_completion it compacts when the history nears the budget;
on :on_context_overflow (the provider rejected the request as too long)
it compacts regardless of the estimate. Either way, when the messages
changed it emits [:alloy, :compaction, :done] and runs the
:after_compaction hook. Every other hook returns the state unchanged.
@spec compact_messages([Alloy.Message.t()], keyword()) :: [Alloy.Message.t()]
Compacts messages, preserving the first message and the most recent N messages.
This helper intentionally stays deterministic and provider-free so it can serve as the fallback truncation strategy.
The preserved window never starts with a tool result: it is extended back to the assistant message that made the call, so it may hold more than N messages.
Options
:keep_recent- minimum number of recent messages to preserve (default 10)
@spec force_compact(Alloy.Agent.State.t(), keyword()) :: Alloy.Agent.State.t()
Forces compaction regardless of reserve budget. Used when the provider rejects the prompt as too long.
The token estimate is not trusted here, because the provider has just rejected a prompt the estimate allowed: clearing old tool results is always followed by summarization (or truncation).
Accepts the same options as maybe_compact/2.
@spec maybe_compact(Alloy.Agent.State.t(), keyword()) :: {:compacted | :unchanged, Alloy.Agent.State.t()}
Compacts state messages when the estimated prompt exceeds
max_tokens - reserve_tokens.
The estimate counts the system prompt, the tool definitions and every message at roughly four bytes per token.
The summary is requested through Alloy.Provider.Retry (retries,
backoff and fallback providers), and its token usage is added to
state.usage.
Options
:turn- turn number reported in compaction telemetry (default:state.turn + 1):deadline- monotonic time in milliseconds by which the summary request must finish (default:state.deadline, or now plusconfig.timeout_mswhen that is unset)
Returns {:compacted, state} when compaction occurred, or
{:unchanged, state} when already within budget.
@spec summary_prefix() :: String.t()
Prefix used for synthetic handoff summary messages inserted by the compactor.