Alloy.Provider behaviour (alloy v0.13.0)

Copy Markdown View Source

Behaviour for LLM providers.

Each provider translates between its native wire format and Alloy's normalized Alloy.Message structs. The agent loop only sees normalized messages - adding a new provider means implementing this behaviour.

Completion Response

Providers return a map with:

  • :stop_reason - why the model stopped (see stop_reason/0)
  • :messages - list of Alloy.Message structs from the response
  • :usage - map with :input_tokens and :output_tokens
  • :provider_state - optional opaque map Alloy feeds back to the same provider on subsequent turns (for example, stored response IDs)
  • :response_metadata - optional provider response metadata exposed to the app layer (for example, citations or server-side tool usage)

Summary

Types

Why the model stopped. Map the wire value to the closest of

Callbacks

Send messages to the provider and get a completion response.

Stream a completion, calling on_chunk for each text delta.

Functions

Decode a JSON binary response body, passing through maps unchanged.

Recursively convert atom keys to strings in maps.

Types

completion_response()

@type completion_response() :: %{
  :stop_reason => stop_reason(),
  :messages => [Alloy.Message.t()],
  :usage => map(),
  optional(:provider_state) => map(),
  optional(:response_metadata) => map()
}

stop_reason()

@type stop_reason() :: :tool_use | :end_turn | :max_tokens | :refusal | :pause_turn

Why the model stopped. Map the wire value to the closest of:

  • :tool_use - the model called client tools; Alloy runs them and loops
  • :end_turn - the model finished (also stop sequences)
  • :max_tokens - output hit the token limit (Anthropic max_tokens or model_context_window_exceeded, OpenAI incomplete/length, Gemini MAX_TOKENS). A truncated answer still completes, with the reason on Alloy.Result; a truncated tool call fails the run because it cannot be executed.
  • :refusal - a safety system stopped the output. Alloy discards the partial output and fails the run. Put the API's own details, with the string keys it sends, in :response_metadata under :stop_details.
  • :pause_turn - the provider paused a long server-side tool loop; Alloy sends the conversation back unchanged so it can continue.

tool_def()

@type tool_def() :: %{name: String.t(), description: String.t(), input_schema: map()}

Callbacks

complete(messages, tool_defs, config)

@callback complete(
  messages :: [Alloy.Message.t()],
  tool_defs :: [tool_def()],
  config :: map()
) ::
  {:ok, completion_response()} | {:error, term()}

Send messages to the provider and get a completion response.

Parameters

  • messages - Conversation history as normalized Alloy.Message structs
  • tool_defs - Tool definitions (JSON Schema format)
  • config - Provider-specific configuration (API keys, model, etc.)

Returns

  • {:ok, completion_response()} on success
  • {:error, term()} on failure

stream(messages, tool_defs, config, on_chunk)

(optional)
@callback stream(
  messages :: [Alloy.Message.t()],
  tool_defs :: [tool_def()],
  config :: map(),
  on_chunk :: (String.t() -> :ok)
) :: {:ok, completion_response()} | {:error, term()}

Stream a completion, calling on_chunk for each text delta.

Returns the same {:ok, completion_response()} as complete/3 once the stream finishes -- the full accumulated response.

Functions

decode_body(body)

@spec decode_body(binary() | map()) :: {:ok, map()} | {:error, String.t()}

Decode a JSON binary response body, passing through maps unchanged.

Returns {:ok, decoded_map} or {:error, reason}.

stringify_keys(map)

@spec stringify_keys(term()) :: term()

Recursively convert atom keys to strings in maps.

Used by providers to prepare JSON-compatible request bodies.