Concepts

Eighteen ideas. Read them once and the rest of the API follows from them. They build on each other roughly in this order, but each page stands alone.

The Request Path

Page	The idea in one line
Targets and aliases	Where a request goes: two spellings that resolve to one thing.
The event stream	A generation is an ordered stream of typed events; everything else is a projection of it.
Routing and rate limits	Retries, fallback chains, health gating, and opt-in pacing; deterministic and fully traceable.
Structured output	A schema is a contract: strongest native mechanism, always client-side validated, optional bounded repair.
Sessions	Letting a provider keep what it already knows, without changing any answer.
Embeddings and reranking	Typed, routed inference operations with a fallback safety rule generation does not need.
Multimodal inputs	Images, documents, audio, and video enter as typed payloads without fictional token estimates.

Cost and Context

Page	The idea in one line
Capabilities and provenance	Every capability value records where it came from, so a developer knows how much to trust it.
Token estimation and context budgets	How many tokens a request will spend, whether it fits, and when to refuse before dispatch.
Cost and spending	Unknown cost stays unknown, while trusted usage and prices support real ceilings.
Prompt caching	Reuse provider-side prompt work without confusing cache hints with guarantees.
Context reduction	Fitting more material than the window holds, and reporting exactly what was dropped.

Local Execution

Page	The idea in one line
The local subsystem	Hardware detection through supervised llama-server, so local models are one target string.
The model catalog	What a machine could run locally, whether this one can run it, and verified acquisition.

Operations

Page	The idea in one line
Credentials and redaction	Secrets are referenced, not embedded, and can never reach a log.
Telemetry and observers	Typed in-process events, payload-free by default.
Run manifests	One serializable, diffable explanation of a call's decisions.
Arena runs	Compare a fixed target set concurrently, select deterministically, and retain every candidate.

The One Rule Underneath All of Them

Adapters only translate. The core orchestrates.

Retry, fallback, health gating, schema validation, repair, TTFT measurement, usage normalization, cost computation, telemetry, and redaction all live in the core, implemented once, behaving identically no matter which provider served the request. A provider adapter does exactly four things: list models, report health, translate a request into its wire format and its responses back into events, and close.

When a developer changes target= from a hosted model to a local one, the behavior the application depends on does not change with it.