Skip to content

Providers

This section is the compatibility inventory: dedicated adapters for protocols that require real translation, plus declarative presets for OpenAI-compatible services and engines. Breadth is useful, but it is not AnyInfer's product boundary; start with when to use AnyInfer if you are choosing an integration layer.

The generated complete inventory records the current counts and target prefixes.

  • OpenAI openai:
    Responses API, reasoning-token accounting.
    Hosted

  • Anthropic anthropic: / claude:
    Messages API, extended thinking deltas.
    Hosted

  • Google Gemini gemini: / google:
    Native generateContent, thinking levels, discovered windows.
    Hosted

  • DeepSeek deepseek:
    Separate reasoning channel, split cache accounting.
    Hosted

  • xAI (Grok) xai: / grok:
    Provider-reported cost, discovered pricing.
    Hosted

  • Google Vertex AI vertex:
    Gemini with GCP auth; project-scoped addressing.
    Enterprise

  • AWS Bedrock bedrock:
    Converse API, SigV4 or API key, binary streaming.
    Enterprise

  • Cohere cohere:
    Native v2 chat, grounded generation, thinking channel.
    Hosted

  • LM Studio lm-studio:
    Native discovery: context, quantization, residency.
    Local

  • Ollama ollama:
    Native API, grammar schemas, phase timings.
    Local

  • llama.cpp llama-cpp:
    Supervised llama-server, loopback only.
    Local

  • OpenAI-compatible openai-compat:
    Any /chat/completions endpoint by URL.
    Local Hosted

  • Hosted & local presets groq: together: mistral: vllm: …
    Eighty-six OpenAI-compatible services and engines, preconfigured.
    Local Hosted

  • OpenRouter openrouter:
    Rich discovered pricing and context data.
    Hosted

  • Nebius Token Factory nebius:
    Live pricing, context, quantization, and reasoning channels.
    Hosted

  • Azure AI Foundry azure-foundry: / azure:
    max_completion_tokens, API key or Entra auth.
    Enterprise

  • GitHub Copilot copilot:
    auto sentinel, CLI-delegated auth.
    Hosted

  • Microsoft 365 Copilot m365-copilot: / m365:
    Interactive auth only, the most constrained provider.
    Enterprise

See the conformance matrix for exactly which behaviors each one supports, generated from test results rather than asserted, and the table below as an accessible alternative to the cards.

Provider Target prefix Extra needed Notes
openai openai: — Responses API
anthropic anthropic: / claude: — Messages API, thinking deltas
gemini gemini: / google: — Native generateContent, thinking levels
deepseek deepseek: — reasoning_content channel, cache split
xai xai: / grok: — Reported cost, discovered pricing
vertex vertex: [vertex] for service-account signing Gemini via GCP OAuth; project-scoped
bedrock bedrock: — Converse API; SigV4 or Bedrock API key
cohere cohere: — Native v2 chat, uppercase enums
lm-studio lm-studio: — Native discovery and residency
ollama ollama: — Native API, grammar schemas, phase timings
llama-cpp llama-cpp: — Supervised llama-server, loopback only
openai-compat openai-compat: — Any /chat/completions endpoint
openrouter openrouter: — Rich discovered pricing and context data
nebius nebius: — Live pricing, context, quantization, reasoning
azure-foundry azure-foundry: / azure: [azure] for Entra max_completion_tokens
copilot copilot: [copilot] auto sentinel, CLI-delegated auth
m365-copilot m365-copilot: / m365: [azure] Interactive auth only

What is the same everywhere

Because the core owns orchestration, these behave identically no matter which provider served a request:

  • retries, fallback, and health gating;
  • structured-output validation and repair;
  • TTFT, duration, and throughput measurement;
  • usage normalization and cost computation;
  • telemetry events and secret redaction;
  • the event stream and its ordering guarantees.

What differs, and how you find out

Real differences are surfaced, never hidden:

  • Capability flags say what a model supports, with provenance.
  • structured_mechanism on each result says how a schema was actually enforced.
  • ParameterDropped events fire when a provider accepts a parameter and discards it.
  • Provider pages document the rest.

The escape hatch

Anything a provider supports that AnyInfer does not model is reachable verbatim:

client.generate(
    prompt,
    target="ollama:qwen3:8b",
    provider_options={"ollama": {"keep_alive": "10m", "num_gpu": 99}},
)

Options are namespaced by provider id and passed straight through to the matching adapter — the core never inspects them. You should never have to fork the library to reach a provider-specific feature.

Adding your own

Third-party adapters register through the anyinfer.providers entry-point group and prove themselves with the same conformance suite the built-ins run. See writing an adapter.