Results and Stream Events¶

The output side: the final Generation, its usage and timing, and the typed events a stream yields on the way there. Ordering guarantees are documented in the event stream.

The Final Result¶

anyinfer.Generation `dataclass` ¶

Generation(
    text: str,
    structured: Any | None,
    tool_calls: tuple[ToolCall, ...],
    target: ResolvedTarget,
    finish_reason: FinishReason,
    usage: Usage,
    timing: Timing,
    structured_mechanism: Mechanism | None = None,
    cache_mechanism: CacheMechanism | None = None,
    repair_attempts: int = 0,
    attempts: tuple[AttemptRecord, ...] = (),
    warnings: tuple[str, ...] = (),
    raw: Any | None = None,
    manifest: RunManifest | None = None,
    arena: ArenaResult | None = None,
    context_reduction: ContextSummary | None = None,
    logprobs: tuple[TokenLogprob, ...] = (),
    citations: tuple[Citation, ...] = (),
    server_tool_uses: tuple[ServerToolUse, ...] = (),
)

The final result of a generation request.

Attributes:

Name	Type	Description
`text`	`str`	The assistant's full text output. Empty when the model answered only with tool calls or structured output.
`structured`	`Any \| None`	The parsed, schema-validated object when structured output was requested; `None` otherwise.
`tool_calls`	`tuple[ToolCall, ...]`	Tool invocations the model requested, in order. Empty when none.
`target`	`ResolvedTarget`	The provider and model that actually produced this result — after routing, so it may differ from the first target asked for.
`finish_reason`	`FinishReason`	Normalized reason the generation stopped.
`usage`	`Usage`	Token accounting, normalized across providers.
`timing`	`Timing`	Centrally-measured latency for the winning attempt.
`structured_mechanism`	`Mechanism \| None`	How structured output was enforced for this result (`grammar`, `json_schema`, `json_mode`, or `prompt`); `None` when no schema was requested.
`cache_mechanism`	`CacheMechanism \| None`	How prompt caching was engaged (`explicit` marks, or `implicit` prefix stability); `None` when no policy was in force or the target offered nothing. Distinct from `usage.cache_read_tokens`, which is what the provider reported — this is what was asked of it.
`repair_attempts`	`int`	How many schema-repair round-trips were needed before `structured` validated. `0` means the first response validated.
`attempts`	`tuple[AttemptRecord, ...]`	The full routing trail, including failed and retried attempts.
`warnings`	`tuple[str, ...]`	Non-fatal notices accumulated along the way (capability downgrades, estimated values, and the like).
`raw`	`Any \| None`	The provider-native response payload, as an escape hatch for fields the normalized types do not carry. `None` unless the request asked to keep it.
`manifest`	`RunManifest \| None`	The run manifest — one content-free record of which target won, which mechanisms were used, what was dropped or reduced, and what it cost. `None` when the client was built with manifests switched off. It is a projection of this call's telemetry events and this result, never an independent account of them.
`arena`	`ArenaResult \| None`	Every arena candidate and the terminal selection, or `None` for an ordinary generation.
`context_reduction`	`ContextSummary \| None`	Content-free account of per-request corpus reduction.
`logprobs`	`tuple[TokenLogprob, ...]`	Per-token log-probabilities for the answer, in generation order, when the request asked for them and the target returned them. Empty otherwise — including for a target that accepted the request and answered without them, which is reported as a dropped parameter rather than inferred from this field.
`server_tool_uses`	`tuple[ServerToolUse, ...]`	How many times the target ran each of its own tools. Empty when none were requested or none ran. Counts only — see `ServerToolUse` for why the queries and results are deliberately absent.
`citations`	`tuple[Citation, ...]`	Attributions the target reported for this answer, in the order it reported them. Empty when none were returned, which includes every target that does not produce them — the presence of citations is a provider capability, never something AnyInfer derives from the text.

anyinfer.Usage `dataclass` ¶

Usage(
    input_tokens: int | None = None,
    output_tokens: int | None = None,
    total_tokens: int | None = None,
    cache_read_tokens: int | None = None,
    cache_write_tokens: int | None = None,
    reasoning_tokens: int | None = None,
    cost_usd: Decimal | None = None,
    search_units: int | None = None,
    server_tool_uses: Mapping[str, int] = dict(),
)

Token accounting for one generation.

Every field is optional: a provider that does not report a number leaves it None rather than reporting a guess.

Attributes:

Name	Type	Description
`input_tokens`	`int \| None`	Tokens in the prompt, as counted by the provider.
`output_tokens`	`int \| None`	Tokens the model generated, including tool-call payloads.
`total_tokens`	`int \| None`	Prompt plus completion tokens; `normalized` fills it from the other two when the provider does not report it.
`cache_read_tokens`	`int \| None`	Prompt tokens served from the provider's prompt cache.
`cache_write_tokens`	`int \| None`	Prompt tokens written into the provider's prompt cache.
`reasoning_tokens`	`int \| None`	Tokens spent on hidden reasoning/thinking, where reported.
`cost_usd`	`Decimal \| None`	Cost of the call in US dollars, computed from per-token pricing when pricing is known.
`search_units`	`int \| None`	Provider-native billed search units (reranking). A distinct billing dimension with its own field on purpose — a search unit is never a token count, and encoding one as the other would fabricate usage.
`server_tool_uses`	`Mapping[str, int]`	Invocations of provider-run tools during this generation, keyed by kind. A third billing dimension for the same reason as the second: a web search is billed per search, so a generation that searched costs more than its token counts say. Carried on usage rather than only on the result so that cost can be computed from usage alone, as every other dimension here is.

normalized ¶

normalized() -> Usage

Fill total_tokens from input + output when both are known.

merge ¶

merge(other: Usage) -> Usage

Overlay other's known fields onto this one.

Later usage reports win; None never overwrites a known value. Streaming providers report usage incrementally, so the router merges rather than replaces. This is an overlay, not addition — to total usage across the internal batches of one request, use Usage.sum().

sum `classmethod` ¶

sum(parts: Sequence[Usage]) -> Usage

Total usage across the internal batches of one logical request.

A field totals only when every part reports it; if any part is unknown, the total stays None — a partial sum would understate spend while reading as authoritative. An empty parts is all-unknown.

anyinfer.Timing `dataclass` ¶

Timing(
    started_at: float,
    first_token_ms: float | None = None,
    total_ms: float = 0.0,
    output_tokens_per_s: float | None = None,
    phases: Mapping[str, float] = dict(),
)

Centrally-measured timings for one attempt.

All values are measured by the core against time.monotonic() so that definitions are identical across providers. phases carries provider-reported sub-timings (e.g. Ollama's model load) in milliseconds.

Attributes:

Name	Type	Description
`started_at`	`float`	Monotonic-clock reading when the attempt began; meaningful only for computing intervals, not as wall-clock time.
`first_token_ms`	`float \| None`	Time to the first content event (text, reasoning, or tool-call delta) after attempt start; `None` when no content ever arrived.
`total_ms`	`float`	Full duration of the attempt, start to completion.
`output_tokens_per_s`	`float \| None`	Decode throughput, measured from first token to completion; `None` when output tokens or first-token time are unknown.
`phases`	`Mapping[str, float]`	Provider-reported sub-timings, keyed by phase name, in milliseconds.

anyinfer.TokenLogprob `dataclass` ¶

TokenLogprob(
    token: str,
    logprob: float,
    top: tuple[TokenLogprob, ...] = (),
    bytes: tuple[int, ...] | None = None,
)

One generated token and how likely the model considered it.

Providers disagree about almost everything here except the two facts that matter — a token and its log-probability — so those are the only required fields. top carries the runners-up when the request asked for alternatives and the provider returned them; an empty tuple means "not asked for, or not answered", which are indistinguishable on the wire and equally uninformative to act on.

Log-probabilities are natural-log values in (-inf, 0], which is what every provider reports and what a caller comparing two of them expects. AnyInfer does not convert them to probabilities: the exponential is one line at the call site and a lossy default here.

Attributes:

Name	Type	Description
`token`	`str`	The generated token, as the provider spelled it.
`logprob`	`float`	Natural log of the token's probability.
`top`	`tuple[TokenLogprob, ...]`	Alternatives the model weighed at this position, most likely first. Empty when none were requested or none were returned.
`bytes`	`tuple[int, ...] \| None`	The token's raw UTF-8 bytes, when the provider reports them. Needed to reassemble text through tokens that split a multi-byte character; `None` when the provider states only the string.

probability `property` ¶

probability: float

The token's probability in [0, 1], exponentiating logprob.

anyinfer.Citation `dataclass` ¶

Citation(
    start_index: int | None = None,
    end_index: int | None = None,
    quoted_text: str = "",
    document_index: int | None = None,
    title: str = "",
    uri: str = "",
)

A span of the answer, and the source material it came from.

The dialects disagree about almost everything: Anthropic reports character offsets into a supplied document plus the exact passage quoted, Cohere reports offsets into the answer plus a source id, and Gemini reports offsets into the answer plus a URI. The union of what they agree is worth saying is: which part of the answer this supports, and enough about the source to show a person. Everything is optional because no dialect fills all of it, and a zero-valued offset is not the same as an absent one — an absent offset means the provider did not say, and rendering a highlight at position zero because of it would be a fabricated claim about the text.

Attributes:

Name	Type	Description
`start_index`	`int \| None`	Character offset into `Generation.text` where the supported span begins, or `None` when the provider located the citation only in the source.
`end_index`	`int \| None`	Exclusive character offset where the supported span ends.
`quoted_text`	`str`	The passage from the source material, when the provider quotes it. Empty when it reports only a location.
`document_index`	`int \| None`	Which of the request's supplied documents this cites, in the order they appeared in the request. `None` for a provider-retrieved source that was never part of the request.
`title`	`str`	Human-readable source name, when the provider supplies one.
`uri`	`str`	Source URL, for providers whose grounding reaches the open web.

span_of ¶

span_of(text: str) -> str

The cited span of an answer, or "" when this citation gives no offsets.

Parameters:

Name	Type	Description	Default
`text`	`str`	The answer text, normally `Generation.text`.	required

Returns:

Type	Description
`str`	The substring the citation supports, clamped to the text's bounds so a
`str`	provider's off-by-one offset yields a short span rather than an exception.

anyinfer.ServerToolUse `dataclass` ¶

ServerToolUse(kind: ServerToolKind, uses: int)

How many times the provider ran one of its own tools during a generation.

A count, not a transcript. What a provider searched for is caller content and the provider's own reasoning about it; carrying either through this library's result type would put prompt-adjacent text somewhere the zero-payload telemetry rules do not reach. The count is what a caller actually needs — these are billed per invocation, so the question the result must answer is "how many did I just pay for?"

Attributes:

Name	Type	Description
`kind`	`ServerToolKind`	Which capability ran.
`uses`	`int`	How many invocations the provider reported.

anyinfer.AttemptRecord `dataclass` ¶

AttemptRecord(
    target: ResolvedTarget,
    outcome: Outcome,
    error: ErrorInfo | None = None,
    timing: Timing | None = None,
)

One entry in a request's routing trail.

Attributes:

Name	Type	Description
`target`	`ResolvedTarget`	The provider and model this attempt was sent to.
`outcome`	`Outcome`	How the attempt ended.
`error`	`ErrorInfo \| None`	Snapshot of the failure, for attempts that did not succeed.
`timing`	`Timing \| None`	Measured timings, when the attempt progressed far enough to have any.

anyinfer.Outcome `module-attribute` ¶

Outcome = Literal[
    "ok",
    "retried",
    "failed",
    "skipped_unhealthy",
    "redirected",
]

How a single routing attempt ended.

redirected marks a completed attempt whose content-filter refusal sent the route to Route.content_policy_targets instead of surfacing the refusal.

anyinfer.ErrorInfo `dataclass` ¶

ErrorInfo(
    type_name: str,
    provider: str | None,
    phase: str,
    retryable: bool,
    http_status: int | None,
    detail: str,
)

A serializable, already-redacted snapshot of any AnyInfer error.

Captured from the exception at failure time, so attempt records and telemetry can carry the failure long after the exception itself has been handled.

Attributes:

Name	Type	Description
`type_name`	`str`	Class name of the exception this snapshot was captured from.
`provider`	`str \| None`	Id of the provider involved, or `None` for provider-independent failures.
`phase`	`str`	Request-lifecycle stage that failed (`configure`, `discover`, `generate`, `stream`, `validate`, or `cleanup`).
`retryable`	`bool`	Whether retrying the identical request could plausibly succeed.
`http_status`	`int \| None`	Status code, for failures that came from an HTTP response.
`detail`	`str`	Human-readable description, already redacted and capped at `DETAIL_MAX_CHARS` characters.

anyinfer.Diagnostic `dataclass` ¶

Diagnostic(
    code: str, severity: DiagnosticSeverity, message: str
)

Something a provider noticed about itself while serving requests.

Not an error and not a capability: an error stops a request, and a capability is a fact about a model. This is the third thing — the request worked, and something about how it worked is worth saying out loud. A model that spilled out of VRAM and is now running at a tenth of its expected speed answers perfectly; the caller simply has no way to know why it took ninety seconds unless the provider says so.

Diagnostics are advisory by construction: collecting them never fails a request, a provider that cannot answer reports nothing, and nothing here is load-bearing for routing. Content-free — a diagnostic describes the runtime, never the prompt.

Attributes:

Name	Type	Description
`code`	`str`	Stable machine-readable identifier, e.g. `"ollama.gpu-spill"`. Callers match on this; the message is for people.
`severity`	`DiagnosticSeverity`	`"warning"` for a condition degrading this request, `"info"` for context that merely explains it.
`message`	`str`	One human-readable sentence, already bounded and redacted.

anyinfer.DiagnosticSeverity `module-attribute` ¶

DiagnosticSeverity = Literal['info', 'warning']

How much a runtime diagnostic should worry the caller. Never an error: a condition that should fail a request is an exception, not a note attached to a successful one.

anyinfer.FinishReason `module-attribute` ¶

FinishReason = Literal[
    "stop",
    "length",
    "tool_calls",
    "content_filter",
    "other",
]

Normalized reason a generation stopped.

anyinfer.ArenaResult `dataclass` ¶

ArenaResult(
    candidates: tuple[Candidate, ...],
    winner: Candidate | None,
    strategy: str,
    agreement: int | None = None,
    synthesized: Generation | None = None,
    calls: int = 0,
    memoized_tool_calls: int = 0,
    usage: Usage = Usage(),
    usage_complete: bool = True,
)

Every candidate, terminal selection, and aggregate accounting for an arena.

summary ¶

summary() -> str

Render one content-free status line.

anyinfer.Candidate `dataclass` ¶

Candidate(
    target: ResolvedTarget,
    generation: Generation | None = None,
    error: ErrorInfo | None = None,
    valid: bool | None = None,
    elapsed_ms: float = 0.0,
    rounds: int | None = None,
    tool_calls: int = 0,
)

One arena branch's final answer or bounded, redacted failure.

anyinfer.TargetComparison `dataclass` ¶

TargetComparison(
    requested: str,
    resolved: ResolvedTarget | None = None,
    resolvable: bool = True,
    reason: str = "",
    fits: bool | None = None,
    budget: ContextBudget | None = None,
    structured_mechanism: Mechanism | None = None,
    mechanism_rungs: tuple[MechanismRung, ...] = (),
    dropped: tuple[DroppedParameter, ...] = (),
    cache: CachePlan | None = None,
    cost: CostEstimate | None = None,
    capability_provenance: Mapping[
        str, Provenance
    ] = dict(),
    notes: tuple[str, ...] = (),
)

What one request would become on one target, without dispatching.

Unresolvable targets are records rather than exceptions. Their target-dependent fields are None and reason says what configuration or identity was missing.

to_dict ¶

to_dict() -> dict[str, Any]

Return a stable JSON-safe representation.

from_dict `classmethod` ¶

from_dict(data: Mapping[str, Any]) -> TargetComparison

Rebuild a comparison produced by to_dict, ignoring unknown keys.

anyinfer.EmbeddingTargetComparison `dataclass` ¶

EmbeddingTargetComparison(
    requested: str,
    resolved: ResolvedTarget | None = None,
    resolvable: bool = True,
    reason: str = "",
    fits: bool | None = None,
    dimensions: int | None = None,
    dimension_choices: tuple[int, ...] = (),
    max_batch_inputs: int | None = None,
    max_input_tokens: int | None = None,
    input_intents: tuple[EmbeddingInputIntent, ...] = (),
    normalized: bool | None = None,
    cost: CostEstimate | None = None,
    capability_provenance: Mapping[
        str, Provenance
    ] = dict(),
    notes: tuple[str, ...] = (),
)

What one embedding request would become on one target, without dispatching.

A separate type from TargetComparison rather than an optional section grafted onto it: generation's dimensions — mechanism rungs, cache planning, structured-output fallback — have no embedding counterpart at all, so folding both into one type would mean every embedding comparison carries a dozen fields that are always None. The dimensions here (space capacity, batch limit, intents, pricing) are what an embedding call actually varies by.

Unresolvable targets are records rather than exceptions, exactly as TargetComparison treats them: target-dependent fields are None/empty and reason says what was missing.

to_dict ¶

to_dict() -> dict[str, Any]

Return a stable JSON-safe representation.

from_dict `classmethod` ¶

from_dict(
    data: Mapping[str, Any],
) -> EmbeddingTargetComparison

Rebuild a comparison produced by to_dict, ignoring unknown keys.

anyinfer.RunManifest `dataclass` ¶

RunManifest(
    format: str = MANIFEST_FORMAT,
    anyinfer_version: str = "",
    request_id: str = "",
    complete: bool = False,
    request: RequestFacet = RequestFacet(),
    route: RouteFacet = RouteFacet(),
    capability: CapabilityFacet = CapabilityFacet(),
    attempts: tuple[AttemptFacet, ...] = (),
    structured: SchemaFacet = SchemaFacet(),
    cache: CacheFacet = CacheFacet(),
    context: ContextFacet = ContextFacet(),
    dropped: tuple[DroppedParameter, ...] = (),
    usage: UsageFacet = UsageFacet(),
    timing: TimingFacet = TimingFacet(),
    notes: tuple[str, ...] = (),
    payloads: PayloadFacet | None = None,
    operation: InferenceOperation = "generation",
    embedding_space: EmbeddingSpace | None = None,
)

One versioned, content-free record of what a single call did.

Attributes:

Name	Type	Description
`format`	`str`	Manifest format version; see `MANIFEST_FORMAT`.
`anyinfer_version`	`str`	The library version that produced it, so a stale record is recognisable as one.
`request_id`	`str`	The correlation id every event for this call carried.
`complete`	`bool`	Whether the call finished. `False` on a manifest read from a cancelled or still-running stream, where the record is a partial account rather than a wrong one.
`request`	`RequestFacet`	Shape and fingerprints of what was asked for.
`route`	`RouteFacet`	Targets requested, considered, and resolved.
`capability`	`CapabilityFacet`	Provenance-tagged capabilities the call consumed.
`attempts`	`tuple[AttemptFacet, ...]`	The routing trail, one entry per attempt.
`structured`	`SchemaFacet`	The structured-output ladder and the repair loop.
`cache`	`CacheFacet`	Prompt-cache plan and reported accounting.
`context`	`ContextFacet`	Reductions applied before dispatch.
`dropped`	`tuple[DroppedParameter, ...]`	Parameters the target would not honour.
`usage`	`UsageFacet`	Token accounting and cost.
`timing`	`TimingFacet`	Latency of the winning attempt.
`notes`	`tuple[str, ...]`	Warnings and provider diagnostics, in the order they arrived.
`payloads`	`PayloadFacet \| None`	Prompt and response text, present only when explicitly asked for.
`operation`	`InferenceOperation`	Which inference operation this record describes. `"generation"` manifests carry every facet; embedding and rerank manifests leave the generation-only facets (`request`, `structured`, `cache`, `context`, `payloads`) at their empty defaults.
`embedding_space`	`EmbeddingSpace \| None`	The vector-space identity an embedding call produced, so an index builder can persist exactly what a stored corpus was embedded with. `None` for every other operation.

to_dict ¶

to_dict() -> dict[str, Any]

Render the manifest as JSON-safe data.

Returns:

Type	Description
`dict[str, Any]`	A plain dictionary of primitives, lists, and dictionaries — directly
`dict[str, Any]`	serializable with `json.dumps`.

from_dict `classmethod` ¶

from_dict(data: Mapping[str, Any]) -> RunManifest

Rebuild a manifest from to_dict output.

Unknown keys are ignored, which is what makes the format's additive rule safe: a reader on an older release still loads a newer manifest.

Parameters:

Name	Type	Description	Default
`data`	`Mapping[str, Any]`	A mapping produced by `to_dict`, or parsed from one.	required

Returns:

Type	Description
`RunManifest`	The reconstructed manifest.

to_json ¶

to_json(*, indent: int | None = 2) -> str

Serialize the manifest to a JSON string.

Parameters:

Name	Type	Description	Default
`indent`	`int \| None`	Indentation passed to `json.dumps`; `None` for the compact form.	`2`

Returns:

Type	Description
`str`	The serialized manifest.

anyinfer.ContextSummary `dataclass` ¶

ContextSummary(
    strategy: str,
    representation: str,
    candidate_count: int,
    selected_count: int,
    omitted_count: int,
    estimated_tokens: int,
    complete: bool,
)

Content-free account of what corpus reduction sent and omitted.

from_reduction `classmethod` ¶

from_reduction(reduction: Reduction) -> ContextSummary

Project a full reduction onto the response-safe summary.

to_dict ¶

to_dict() -> dict[str, object]

Serialize for the sidecar extension.

anyinfer.MANIFEST_FORMAT `module-attribute` ¶

MANIFEST_FORMAT = '1'

Manifest format version.

Bumped when an existing field's meaning changes, never when a field is added — a reader that ignores unknown keys survives additions, which is the same rule the context envelope follows.

anyinfer.RequestFacet `dataclass` ¶

RequestFacet(
    message_count: int = 0,
    role_counts: Mapping[str, int] = dict(),
    char_count: int = 0,
    estimated_tokens: int | None = None,
    schema_present: bool = False,
    schema_name: str | None = None,
    schema_digest: str | None = None,
    tool_names: tuple[str, ...] = (),
    tool_choice: str = "auto",
    sampling: Mapping[str, Any] = dict(),
    reasoning: str | None = None,
    timeout_s: float | None = None,
    repair_budget: int = 0,
    metadata_keys: tuple[str, ...] = (),
)

The shape and fingerprints of what was asked for, never the payload.

Attributes:

Name	Type	Description
`message_count`	`int`	How many messages the request carried.
`role_counts`	`Mapping[str, int]`	Message count per role, in role order.
`char_count`	`int`	Total characters of message text.
`estimated_tokens`	`int \| None`	Planning-side input estimate, or `None` when not computed.
`schema_present`	`bool`	Whether structured output was requested.
`schema_name`	`str \| None`	The schema's label, which is a title rather than content.
`schema_digest`	`str \| None`	SHA-256 of the canonical schema JSON, so two runs can be compared without either one carrying the schema body.
`tool_names`	`tuple[str, ...]`	Names of the tools offered, in order.
`tool_choice`	`str`	How tool use was constrained.
`sampling`	`Mapping[str, Any]`	Sampling controls actually set, omitting the ones left unset.
`reasoning`	`str \| None`	Requested reasoning effort, when one was asked for.
`timeout_s`	`float \| None`	Per-attempt wall clock the request carried, when set.
`repair_budget`	`int`	Repair round-trips the request allowed.
`metadata_keys`	`tuple[str, ...]`	Keys of caller-supplied metadata; values may be anything.

anyinfer.RouteFacet `dataclass` ¶

RouteFacet(
    requested: tuple[str, ...] = (),
    resolved: str | None = None,
    considered: tuple[RouteStep, ...] = (),
)

Which targets were asked for, which one answered, and what happened between.

Attributes:

Name	Type	Description
`requested`	`tuple[str, ...]`	The fallback chain as the caller wrote it, unresolved.
`resolved`	`str \| None`	The target that produced the result, or `None` when none did.
`considered`	`tuple[RouteStep, ...]`	Every target the router touched, in the order it touched them.

anyinfer.RouteStep `dataclass` ¶

RouteStep(target: str, outcome: str, reason: str = '')

One target the router considered, and what became of it.

Attributes:

Name	Type	Description
`target`	`str`	The resolved target, as `provider:model`.
`outcome`	`str`	`ok`, `failed`, `retried`, `skipped_unhealthy`, `redirected`, or `abandoned` for a target the route left before it produced a result.
`reason`	`str`	Why, in the words the router used — a health gate, a context overflow, a content-policy redirect, or the error that ended it.

anyinfer.CapabilityFacet `dataclass` ¶

CapabilityFacet(
    target: str = "", facts: tuple[SourcedFact, ...] = ()
)

Every provenance-tagged capability the call actually consumed.

Attributes:

Name	Type	Description
`target`	`str`	The target these capabilities describe.
`facts`	`tuple[SourcedFact, ...]`	One entry per known capability field, provenance intact.

anyinfer.SourcedFact `dataclass` ¶

SourcedFact(name: str, value: Any, provenance: str)

One capability value the call consumed, with its provenance intact.

Provenance is carried verbatim rather than collapsed into a bare value: "the context window was 8192" and "the context window was assumed to be 8192" are different statements, and a manifest that could not tell them apart would be useless for the question it exists to answer.

Attributes:

Name	Type	Description
`name`	`str`	Which capability this is, e.g. `context_window`.
`value`	`Any`	The value itself, rendered as JSON-safe data.
`provenance`	`str`	Where it came from — `catalog`, `discovered`, `probed`, `default`, or `override`.

anyinfer.AttemptFacet `dataclass` ¶

AttemptFacet(
    target: str,
    attempt_number: int = 1,
    outcome: str = "ok",
    error: ErrorInfo | None = None,
    first_token_ms: float | None = None,
    total_ms: float | None = None,
    queued_ms: float | None = None,
    retry_reason: str | None = None,
    retry_delay_s: float | None = None,
    paced_s: Mapping[str, float] = dict(),
)

One attempt against one target, with the reason it ended as it did.

Attributes:

Name	Type	Description
`target`	`str`	The resolved target attempted.
`attempt_number`	`int`	1-based count against this target; a fallback restarts it at 1.
`outcome`	`str`	How it ended, using the same vocabulary as `anyinfer.AttemptRecord`.
`error`	`ErrorInfo \| None`	The failure snapshot, for an attempt that did not succeed.
`first_token_ms`	`float \| None`	Time to the first content delta, when any arrived.
`total_ms`	`float \| None`	Full duration of the attempt.
`queued_ms`	`float \| None`	How long client-side pacing held it before dispatch.
`retry_reason`	`str \| None`	Why a retry was scheduled after this attempt, when one was.
`retry_delay_s`	`float \| None`	How long the router waited before retrying.
`paced_s`	`Mapping[str, float]`	Seconds this attempt spent waiting on a rate limiter, and why, summed per reason.

anyinfer.SchemaFacet `dataclass` ¶

SchemaFacet(
    requested: bool = False,
    chosen: str | None = None,
    used: str | None = None,
    ladder: tuple[MechanismRung, ...] = (),
    repair_attempts: int = 0,
    repairs: tuple[RepairRecord, ...] = (),
    validated: bool = False,
)

What was asked of the structured-output ladder, and what it delivered.

Attributes:

Name	Type	Description
`requested`	`bool`	Whether the request carried a schema at all.
`chosen`	`str \| None`	The mechanism the ladder selected before dispatch.
`used`	`str \| None`	The mechanism the winning attempt actually used.
`ladder`	`tuple[MechanismRung, ...]`	Every rung considered, strongest first, with the reason each was rejected.
`repair_attempts`	`int`	How many repair round-trips were spent.
`repairs`	`tuple[RepairRecord, ...]`	One record per repair, with the validation errors that caused it.
`validated`	`bool`	Whether a structured value was finally produced.

anyinfer.MechanismRung `dataclass` ¶

MechanismRung(
    mechanism: str, available: bool, reason: str = ""
)

One structured-output rung and why it was or was not selected.

anyinfer.CacheFacet `dataclass` ¶

CacheFacet(
    policy_mode: str | None = None,
    mechanism: str | None = None,
    mark_count: int = 0,
    estimated_cacheable_tokens: int = 0,
    read_tokens: int | None = None,
    write_tokens: int | None = None,
)

What was planned for the target's prompt cache, and what it reported back.

read_tokens and write_tokens are what the provider said; everything else is what was asked of it. The two are kept apart because an intention is not a saving.

Attributes:

Name	Type	Description
`policy_mode`	`str \| None`	The cache mode in force, or `None` when no policy applied.
`mechanism`	`str \| None`	How caching was engaged: `explicit` marks or `implicit` prefix stability. `None` means caching was not engaged.
`mark_count`	`int`	How many marks were placed; always zero for `implicit`.
`estimated_cacheable_tokens`	`int`	Planning-side size of what the plan tried to cache.
`read_tokens`	`int \| None`	Prompt tokens the provider reported serving from its cache.
`write_tokens`	`int \| None`	Prompt tokens the provider reported writing into its cache.

anyinfer.ContextFacet `dataclass` ¶

ContextFacet(reductions: tuple[ReductionRecord, ...] = ())

Reductions the request went through before it was sent.

Attributes:

Name	Type	Description
`reductions`	`tuple[ReductionRecord, ...]`	One record per reduction, in the order they were applied.

anyinfer.ReductionRecord `dataclass` ¶

ReductionRecord(
    strategy: str,
    representation: str,
    candidate_count: int = 0,
    selected_count: int = 0,
    omitted_count: int = 0,
    estimated_tokens: int = 0,
    max_tokens: int = 0,
    binding_constraints: tuple[str, ...] = (),
    calls: int = 0,
    complete: bool = True,
)

One context reduction applied on the way to dispatch.

Attributes:

Name	Type	Description
`strategy`	`str`	The strategy requested, or `history` for a compacted conversation.
`representation`	`str`	The strategy actually applied.
`candidate_count`	`int`	Documents or messages offered to the reducer.
`selected_count`	`int`	Documents kept at detail fidelity, or messages retained.
`omitted_count`	`int`	What was not represented in detail.
`estimated_tokens`	`int`	Planning-side estimate of the result.
`max_tokens`	`int`	The budget the reduction was held to.
`binding_constraints`	`tuple[str, ...]`	Which ceilings excluded at least one candidate.
`calls`	`int`	Generation calls the reduction itself spent.
`complete`	`bool`	Whether nothing was omitted.

anyinfer.RepairRecord `dataclass` ¶

RepairRecord(
    attempt_number: int,
    mechanism: str | None = None,
    errors: tuple[str, ...] = (),
)

One schema-repair round trip.

Attributes:

Name	Type	Description
`attempt_number`	`int`	1-based repair count within this generation.
`mechanism`	`str \| None`	The mechanism in force when validation failed.
`errors`	`tuple[str, ...]`	The validation messages that triggered the repair.

anyinfer.DroppedParameter `dataclass` ¶

DroppedParameter(target: str, parameter: str, reason: str)

A requested parameter the target would not honour as asked.

Attributes:

Name	Type	Description
`target`	`str`	Which target withheld it.
`parameter`	`str`	The parameter name, dotted for a field of a compound one.
`reason`	`str`	What the target did instead.

anyinfer.UsageFacet `dataclass` ¶

UsageFacet(
    input_tokens: int | None = None,
    output_tokens: int | None = None,
    total_tokens: int | None = None,
    cache_read_tokens: int | None = None,
    cache_write_tokens: int | None = None,
    reasoning_tokens: int | None = None,
    cost_usd: str | None = None,
    estimated_fields: Mapping[str, str] = dict(),
    search_units: int | None = None,
)

Token accounting and cost, with estimated figures marked as estimated.

Attributes:

Name	Type	Description
`input_tokens`	`int \| None`	Prompt tokens, as counted by the provider.
`output_tokens`	`int \| None`	Generated tokens.
`total_tokens`	`int \| None`	Prompt plus completion.
`cache_read_tokens`	`int \| None`	Prompt tokens served from the provider's cache.
`cache_write_tokens`	`int \| None`	Prompt tokens written into it.
`reasoning_tokens`	`int \| None`	Tokens spent on hidden reasoning, where reported.
`cost_usd`	`str \| None`	Cost as a decimal string, or `None` when pricing is not trustworthy for this target. Never zero for an unpriced call.
`estimated_fields`	`Mapping[str, str]`	Usage fields that were derived rather than reported, each with the method used.
`search_units`	`int \| None`	Provider-native billed search units (reranking), where reported.

anyinfer.TimingFacet `dataclass` ¶

TimingFacet(
    first_token_ms: float | None = None,
    total_ms: float | None = None,
    output_tokens_per_s: float | None = None,
    phases: Mapping[str, float] = dict(),
)

Centrally-measured latency for the winning attempt.

Attributes:

Name	Type	Description
`first_token_ms`	`float \| None`	Time to the first content delta.
`total_ms`	`float \| None`	Full duration.
`output_tokens_per_s`	`float \| None`	Decode throughput.
`phases`	`Mapping[str, float]`	Provider-reported sub-timings, in milliseconds.

anyinfer.PayloadFacet `dataclass` ¶

PayloadFacet(
    prompt_text: str | None = None,
    response_text: str | None = None,
    schema_body: str | None = None,
    tool_arguments: tuple[str, ...] = (),
    repair_texts: tuple[str, ...] = (),
)

The strings a content-free manifest deliberately leaves out.

Populated only when a manifest is built with payloads enabled, and every value here has already passed through the redaction registry. The default manifest carries None in its place, which is what makes "safe to paste into a public issue tracker" a structural property rather than a promise about field contents.

Attributes:

Name	Type	Description
`prompt_text`	`str \| None`	The request's message text, flattened.
`response_text`	`str \| None`	The final response text.
`schema_body`	`str \| None`	The JSON Schema the request carried.
`tool_arguments`	`tuple[str, ...]`	Arguments of each tool call the model requested, as JSON.
`repair_texts`	`tuple[str, ...]`	The responses that failed validation, in repair order.

anyinfer.manifest_json_schema ¶

manifest_json_schema() -> dict[str, Any]

The JSON Schema a serialized manifest validates against.

Published so a golden-file workflow has something to check against, and marked pre-1.0 alongside the rest of this API. Readers must ignore unknown keys: the format adds fields without a version bump, and only a change of meaning bumps MANIFEST_FORMAT.

Returns:

Type	Description
`dict[str, Any]`	The schema as a plain dictionary.

anyinfer.context.ContextSummary `dataclass` ¶

ContextSummary(
    strategy: str,
    representation: str,
    candidate_count: int,
    selected_count: int,
    omitted_count: int,
    estimated_tokens: int,
    complete: bool,
)

Content-free account of what corpus reduction sent and omitted.

from_reduction `classmethod` ¶

from_reduction(reduction: Reduction) -> ContextSummary

Project a full reduction onto the response-safe summary.

to_dict ¶

to_dict() -> dict[str, object]

Serialize for the sidecar extension.

Stream Events¶

anyinfer.StreamEvent `module-attribute` ¶

StreamEvent = (
    TextDelta
    | ReasoningDelta
    | ToolCallDelta
    | CitationDelta
    | ServerToolDelta
    | UsageUpdate
    | TimingMark
    | AttemptFailed
    | StreamEnded
)

Any event a consumer may observe.

anyinfer.TextDelta `dataclass` ¶

TextDelta(text: str)

A fragment of visible answer text.

anyinfer.ReasoningDelta `dataclass` ¶

ReasoningDelta(text: str)

A fragment of reasoning/thinking text, excluded from the answer text.

anyinfer.ToolCall `dataclass` ¶

ToolCall(id: str, name: str, arguments: Mapping[str, Any])

A model's request to invoke a tool.

Attributes:

Name	Type	Description
`id`	`str`	Provider-assigned call id. Adapters synthesize `"call_0"`, `"call_1"`… when the provider omits one, so downstream correlation always has a key.
`name`	`str`	The tool being called.
`arguments`	`Mapping[str, Any]`	Parsed JSON arguments. An unparseable argument payload yields `{}` and a warning on the `Generation`.

anyinfer.ToolCallDelta `dataclass` ¶

ToolCallDelta(
    index: int,
    call_id: str | None,
    name: str | None,
    arguments_fragment: str,
)

A fragment of a tool call.

Fragments are correlated by index — the tool-call slot within the response. Concatenate arguments_fragment per index, then JSON-parse the result.

Attributes:

Name	Type	Description
`index`	`int`	The tool-call slot within the response this fragment belongs to.
`call_id`	`str \| None`	Provider-assigned call id, on fragments that carry it.
`name`	`str \| None`	Name of the tool being called, on fragments that carry it.
`arguments_fragment`	`str`	The next piece of this slot's JSON argument text; may be empty.

anyinfer.CitationDelta `dataclass` ¶

CitationDelta(citation: Citation)

One attribution the model reported, as soon as it reported it.

Named a delta to match its siblings even though a citation arrives whole: it is a stream event that adds to the answer, and callers rendering attributions live need it at the moment it lands rather than at StreamEnded. Every citation on the terminal result also appeared here first, so a consumer may use either and never both.

Deliberately not a content event: a citation does not start the first-token clock. Cohere emits its first citation only after the span it supports, so counting one as first content would report a time-to-first-token later than the text the user already saw.

Attributes:

Name	Type	Description
`citation`	`Citation`	The attribution, with whatever the dialect reported about it.

anyinfer.ServerToolDelta `dataclass` ¶

ServerToolDelta(
    kind: ServerToolKind,
    status: ServerToolStatus,
    sources: tuple[ServerToolSource, ...] = (),
    output: str = "",
    detail: str = "",
)

A provider-run tool started, or finished and returned something.

Emitted so a caller watching a stream can say why an answer paused — a web search takes seconds, and a stream that stops producing text for that long is otherwise indistinguishable from a stalled connection — and, on completion, what it found.

Carrying the result is the point rather than a nicety. "Grounded answer with fresh web results" is the application feature this exists for, and an application that can render the answer but not the sources behind it has the less useful half. This is a stream event, on the content channel beside TextDelta, so carrying content is what it is for; the payload-free rule governs telemetry events, which these are not.

Deliberately not a content event, so it does not start the first-token clock: a provider that searches before writing anything has not produced a token yet, and counting one would make time-to-first-token mean something different for those requests than for every other.

Attributes:

Name	Type	Description
`kind`	`ServerToolKind`	Which capability ran.
`status`	`ServerToolStatus`	Where in its lifecycle it is.
`sources`	`tuple[ServerToolSource, ...]`	What a search returned, in the order the provider listed it. Empty on a `started` event, and for kinds that return no sources.
`output`	`str`	What a code execution printed. Empty when there was none.
`detail`	`str`	Why a `failed` invocation failed, as the provider stated it.

anyinfer.ServerToolSource `dataclass` ¶

ServerToolSource(url: str = '', title: str = '')

One source a provider-run search consulted.

Attributes:

Name	Type	Description
`url`	`str`	Where it came from.
`title`	`str`	The page's title, when the provider reports one.

anyinfer.UsageUpdate `dataclass` ¶

UsageUpdate(usage: Usage)

A usage report; may arrive mid-stream and more than once.

anyinfer.TimingMark `dataclass` ¶

TimingMark(name: TimingMarkName, at_ms: float)

A centrally-measured timing point, in milliseconds since attempt start.

Attributes:

Name	Type	Description
`name`	`TimingMarkName`	Which point on the attempt clock this marks.
`at_ms`	`float`	Milliseconds elapsed since the attempt started.

anyinfer.TimingMarkName `module-attribute` ¶

TimingMarkName = Literal['attempt_start', 'first_token']

Named points on the attempt clock.

anyinfer.AttemptFailed `dataclass` ¶

AttemptFailed(record: AttemptRecord)

A target attempt failed; a retry or fallback may follow.

Attributes:

Name	Type	Description
`record`	`AttemptRecord`	The failed attempt's routing-trail entry: target, outcome, error snapshot, and any timing.

anyinfer.StreamEnded `dataclass` ¶

StreamEnded(result: Generation)

Terminal event carrying the assembled result.

Results and Stream Events¶

The Final Result¶

anyinfer.Generation dataclass ¶

anyinfer.Usage dataclass ¶

normalized ¶

merge ¶

sum classmethod ¶

anyinfer.Timing dataclass ¶

anyinfer.TokenLogprob dataclass ¶

probability property ¶

anyinfer.Citation dataclass ¶

span_of ¶

anyinfer.ServerToolUse dataclass ¶

anyinfer.AttemptRecord dataclass ¶

anyinfer.Outcome module-attribute ¶

anyinfer.ErrorInfo dataclass ¶

anyinfer.Diagnostic dataclass ¶

anyinfer.DiagnosticSeverity module-attribute ¶

anyinfer.FinishReason module-attribute ¶

anyinfer.ArenaResult dataclass ¶

summary ¶

anyinfer.Candidate dataclass ¶

anyinfer.TargetComparison dataclass ¶

to_dict ¶

from_dict classmethod ¶

anyinfer.EmbeddingTargetComparison dataclass ¶

to_dict ¶

from_dict classmethod ¶

anyinfer.RunManifest dataclass ¶

to_dict ¶

from_dict classmethod ¶

to_json ¶

anyinfer.ContextSummary dataclass ¶

from_reduction classmethod ¶

to_dict ¶

Run Manifest Facets¶

anyinfer.MANIFEST_FORMAT module-attribute ¶

anyinfer.RequestFacet dataclass ¶

anyinfer.RouteFacet dataclass ¶

anyinfer.RouteStep dataclass ¶

anyinfer.CapabilityFacet dataclass ¶

anyinfer.SourcedFact dataclass ¶

anyinfer.AttemptFacet dataclass ¶

anyinfer.SchemaFacet dataclass ¶

anyinfer.MechanismRung dataclass ¶

anyinfer.CacheFacet dataclass ¶

anyinfer.ContextFacet dataclass ¶

anyinfer.ReductionRecord dataclass ¶

anyinfer.RepairRecord dataclass ¶

anyinfer.DroppedParameter dataclass ¶

anyinfer.UsageFacet dataclass ¶

anyinfer.TimingFacet dataclass ¶

anyinfer.PayloadFacet dataclass ¶

anyinfer.manifest_json_schema ¶

anyinfer.context.ContextSummary dataclass ¶

from_reduction classmethod ¶

to_dict ¶

Stream Events¶

anyinfer.StreamEvent module-attribute ¶

anyinfer.TextDelta dataclass ¶

anyinfer.ReasoningDelta dataclass ¶

anyinfer.ToolCall dataclass ¶

anyinfer.ToolCallDelta dataclass ¶

anyinfer.CitationDelta dataclass ¶

anyinfer.ServerToolDelta dataclass ¶

anyinfer.ServerToolSource dataclass ¶

anyinfer.UsageUpdate dataclass ¶

anyinfer.TimingMark dataclass ¶

anyinfer.TimingMarkName module-attribute ¶

anyinfer.AttemptFailed dataclass ¶

anyinfer.StreamEnded dataclass ¶

anyinfer.Generation `dataclass` ¶

anyinfer.Usage `dataclass` ¶

sum `classmethod` ¶

anyinfer.Timing `dataclass` ¶

anyinfer.TokenLogprob `dataclass` ¶

probability `property` ¶

anyinfer.Citation `dataclass` ¶

anyinfer.ServerToolUse `dataclass` ¶

anyinfer.AttemptRecord `dataclass` ¶

anyinfer.Outcome `module-attribute` ¶

anyinfer.ErrorInfo `dataclass` ¶

anyinfer.Diagnostic `dataclass` ¶

anyinfer.DiagnosticSeverity `module-attribute` ¶

anyinfer.FinishReason `module-attribute` ¶

anyinfer.ArenaResult `dataclass` ¶

anyinfer.Candidate `dataclass` ¶

anyinfer.TargetComparison `dataclass` ¶

from_dict `classmethod` ¶

anyinfer.EmbeddingTargetComparison `dataclass` ¶

from_dict `classmethod` ¶

anyinfer.RunManifest `dataclass` ¶

from_dict `classmethod` ¶

anyinfer.ContextSummary `dataclass` ¶

from_reduction `classmethod` ¶

anyinfer.MANIFEST_FORMAT `module-attribute` ¶

anyinfer.RequestFacet `dataclass` ¶

anyinfer.RouteFacet `dataclass` ¶

anyinfer.RouteStep `dataclass` ¶

anyinfer.CapabilityFacet `dataclass` ¶

anyinfer.SourcedFact `dataclass` ¶

anyinfer.AttemptFacet `dataclass` ¶

anyinfer.SchemaFacet `dataclass` ¶

anyinfer.MechanismRung `dataclass` ¶

anyinfer.CacheFacet `dataclass` ¶

anyinfer.ContextFacet `dataclass` ¶

anyinfer.ReductionRecord `dataclass` ¶

anyinfer.RepairRecord `dataclass` ¶

anyinfer.DroppedParameter `dataclass` ¶

anyinfer.UsageFacet `dataclass` ¶

anyinfer.TimingFacet `dataclass` ¶

anyinfer.PayloadFacet `dataclass` ¶

anyinfer.context.ContextSummary `dataclass` ¶

from_reduction `classmethod` ¶

anyinfer.StreamEvent `module-attribute` ¶

anyinfer.TextDelta `dataclass` ¶

anyinfer.ReasoningDelta `dataclass` ¶

anyinfer.ToolCall `dataclass` ¶

anyinfer.ToolCallDelta `dataclass` ¶

anyinfer.CitationDelta `dataclass` ¶

anyinfer.ServerToolDelta `dataclass` ¶

anyinfer.ServerToolSource `dataclass` ¶

anyinfer.UsageUpdate `dataclass` ¶

anyinfer.TimingMark `dataclass` ¶

anyinfer.TimingMarkName `module-attribute` ¶

anyinfer.AttemptFailed `dataclass` ¶

anyinfer.StreamEnded `dataclass` ¶