Cohere¶

The native v2 Chat API, chosen over the OpenAI compatibility layer because v2 is where the things worth choosing Cohere for live: grounded generation with document citations, a separate thinking channel, and usage that distinguishes what was processed from what was billed.

streaming structured output tool calls reasoning discovery (context lengths)

Setup¶

import anyinfer as ai

client = ai.Client(
    [
        ai.ProviderSettings.of("cohere", api_key="env://CO_API_KEY"),
    ]
)

result = client.generate(prompt, target="cohere:command-a-03-2025")

Dialect Differences¶

Cohere's API diverges from the OpenAI shape in ways the adapter normalizes, but which show up when reaching past it with provider_options:

AnyInfer	Cohere
`finish_reason == "stop"`	`COMPLETE` (uppercase enum)
`tool_choice="required"`	`"REQUIRED"`; there is no way to name one tool
`Sampling(top_p=...)`	`p`, not `top_p`
optional streaming	`stream` is required on every request

Unknown finish reasons normalize to "other" rather than propagating.

Reasoning¶

Cohere budgets thinking in tokens, so normalized effort maps to a budget: minimal disables thinking, and low/medium/high map to increasing token budgets. Thinking blocks arrive as ReasoningDelta events on the event stream and stay out of result.text.

result = client.generate(prompt, target="cohere:command-a-03-2025", reasoning="high")

Usage Accounting¶

Cohere reports both billed_units and tokens. AnyInfer's counts follow tokens (what the model actually processed, which is what a context window measures):

result = client.generate(prompt, target="cohere:command-a-03-2025")
print(result.usage.input_tokens)  # processed

If billed units are needed for cost reconciliation, build the client with retain_raw=True and read them off result.raw.

Grounded Generation¶

client.generate(
    question,
    target="cohere:command-a-03-2025",
    provider_options={
        "cohere": {
            "documents": [{"id": "doc1", "data": {"text": "..."}}],
            "citation_options": {"mode": "ACCURATE"},
        }
    },
)

Document grounding and citations are reachable through the escape hatch. Citations are not yet surfaced as typed results; read them from result.raw until they are modeled.

Embeddings and Reranking¶

Cohere serves both operations natively (POST /v2/embed, POST /v2/rerank), and is the first provider here with native input intents and native rerank scores:

docs = client.embed(
    ["the cat sat on the mat", "stock markets rallied"],
    target="cohere:embed-v4.0",
    input_type="document",
)
ranked = client.rerank(
    "where did the cat sit",
    ["stock markets rallied", "the cat sat on the mat"],
    target="cohere:rerank-v3.5",
    top_n=1,
)

Three things worth knowing:

input_type is required. Cohere's embed API demands an intent and documents no default, so an intent-less embed() is refused with a hint rather than guessed; query and document embeddings are not comparable unless produced with matching intents.
Batching engages at 96 inputs. The endpoint accepts at most 96 texts per call; larger requests are split by the core and re-assembled in input order, invisibly. Requested dimensions are forwarded as output_dimension (embed-v4 models only).
Rerank usage is search units, not tokens. Live rerank responses report only billed_units.search_units, which AnyInfer never encodes as fake token counts, so result.usage is typically empty for rerank. The billed units are on result.raw["meta"]["billed_units"] for cost reconciliation.

Discovery¶

The model listing reports real context lengths, so windows carry discovered provenance:

for model in client.models("cohere"):
    caps = model.capabilities
    if caps and caps.context_window:
        print(model.id, caps.context_window.value, caps.context_window.provenance)

Every model is listed (embedding and rerank models included), with its operations derived from the listing's endpoints field, so client.models("cohere", operation="embedding") answers from discovery rather than a guess.

Wire Contract¶

For the exact request/response fields this adapter depends on, see contracts/cohere.md.