Alloy.Provider.OpenAICompat (alloy v0.13.0)

Copy Markdown View Source

Generic OpenAI-compatible provider.

Works with any API that implements the OpenAI chat completions format: DeepSeek, Mistral, Kimi, GLM, Ollama, OpenRouter, Together, Groq, etc.

Reasoning returned as reasoning_content (DeepSeek, Kimi, GLM) becomes a "thinking" block and is sent back on later assistant messages, which DeepSeek's thinking mode requires during tool calls.

Two providers have better native options:

  • OpenAI: use Alloy.Provider.OpenAI. Starting with GPT-5.4, Chat Completions does not support tool calling with reasoning enabled.
  • xAI: use Alloy.Provider.XAI. xAI's Chat Completions endpoint is deprecated and returns no reasoning content.

Config

Required:

Optional:

  • :api_key - API key (omit for local providers like Ollama)
  • :max_tokens - Max output tokens. Omitted unless set, so the server's default applies
  • :system_prompt - System prompt string
  • :chat_path - Path to completions endpoint (default: "/v1/chat/completions")
  • :extra_headers - Additional headers as [{name, value}]
  • :extra_body - Additional body fields, merged last
  • :stream_options - Set to false to omit usage-request options from streaming requests. Custom option maps can be supplied through extra_body: %{"stream_options" => options}.
  • :req_options - Additional options passed to Req

Examples

# DeepSeek
Alloy.run("Hello",
  provider: {Alloy.Provider.OpenAICompat,
    api_key: System.get_env("DEEPSEEK_API_KEY"),
    api_url: "https://api.deepseek.com",
    model: "deepseek-v4-pro"
  }
)

# Ollama (no API key)
Alloy.run("Hello",
  provider: {Alloy.Provider.OpenAICompat,
    api_url: "http://localhost:11434",
    model: "llama4"
  }
)

# Mistral (mistral-large-latest and -2512 are in the built-in
# model catalog, so context budgeting works out of the box)
Alloy.run("Hello",
  provider: {Alloy.Provider.OpenAICompat,
    api_key: System.get_env("MISTRAL_API_KEY"),
    api_url: "https://api.mistral.ai",
    model: "mistral-large-latest"
  }
)

Summary

Types

Configuration for the OpenAI-compatible provider. :api_key is optional (omit for local providers like Ollama). See the module doc for field semantics.

Types

config()

@type config() :: %{
  :api_url => String.t(),
  :model => String.t(),
  optional(:api_key) => String.t(),
  optional(:max_tokens) => pos_integer(),
  optional(:system_prompt) => String.t(),
  optional(:chat_path) => String.t(),
  optional(:extra_headers) => [{String.t(), String.t()}],
  optional(:extra_body) => map(),
  optional(:stream_options) => false,
  optional(:req_options) => keyword()
}

Configuration for the OpenAI-compatible provider. :api_key is optional (omit for local providers like Ollama). See the module doc for field semantics.