Capabilities

Provenance-tagged model metadata: every value knows whether it was catalogued, discovered, probed, or defaulted. The reasoning is in capabilities and provenance; token estimation and the budget calculator are explained in token estimation and context budgets.

anyinfer.ModelCapabilities dataclass

ModelCapabilities(
    context_window: Sourced[int] | None = None,
    max_output_tokens: Sourced[int] | None = None,
    features: Sourced[Feature] = Sourced(
        Feature(0), "default"
    ),
    pricing: Sourced[Pricing] | None = None,
    local: LocalModelInfo | None = None,
)

What a model can do, as far as we know.

Attributes:

Name	Type	Description
context_window	Sourced[int] | None	Maximum tokens of input context, with provenance; None when unknown.
max_output_tokens	Sourced[int] | None	Maximum tokens one response may contain, with provenance; None when unknown.
features	Sourced[Feature]	Which Feature flags the model supports, with provenance.
pricing	Sourced[Pricing] | None	Per-million-token pricing, when known.
local	LocalModelInfo | None	Facts about the local artifact, for locally-run models only.

overlay

overlay(other: ModelCapabilities) -> ModelCapabilities

Layer other on top of this, field by field, stronger provenance winning.

This is the assembly rule: later layers override earlier ones, but a weaker-provenance value never displaces a stronger one.

anyinfer.Pricing dataclass

Pricing(
    input_per_1m: Decimal,
    output_per_1m: Decimal,
    currency: str = "USD",
)

Per-million-token pricing used to compute cost_usd.

Attributes:

Name	Type	Description
input_per_1m	Decimal	Price per one million prompt tokens.
output_per_1m	Decimal	Price per one million generated tokens.
currency	str	Currency code the prices are quoted in.

anyinfer.Feature

Bases: Flag

Capabilities a model may support.

Structured-output mechanism selection reads these in the order GRAMMAR > JSON_SCHEMA > JSON_MODE > prompt injection.

anyinfer.Mechanism module-attribute

Mechanism = Literal[
    "grammar", "json_schema", "json_mode", "prompt"
]

How structured output was requested of the provider.

anyinfer.Sourced dataclass

Sourced(value: _T, provenance: Provenance = 'default')

Bases: Generic[_T]

A capability value paired with its provenance.

outranks

outranks(other: Sourced[_T] | None) -> bool

Whether this value's provenance is at least as strong as other's.

anyinfer.Provenance module-attribute

Provenance = Literal[
    "catalog", "discovered", "probed", "default", "override"
]

Where a capability value came from, weakest (default) to strongest (override).

anyinfer.Health dataclass

Health(ok: bool, detail: str = '')

Result of a provider's cheap readiness probe.

Attributes:

Name	Type	Description
ok	bool	Whether the provider answered its readiness probe successfully.
detail	str	Short human-readable explanation, most useful when ok is false.

anyinfer.DiscoveredModel dataclass

DiscoveredModel(
    id: str, capabilities: ModelCapabilities | None = None
)

A model reported by a provider's listing endpoint.

capabilities carries only fields the provider actually reported; the capability assembler tags them "discovered".

Attributes:

Name	Type	Description
id	str	The model identifier exactly as the provider lists it.
capabilities	ModelCapabilities | None	Capability fields the listing reported; None when the provider lists ids only.

anyinfer.LocalModelInfo dataclass

LocalModelInfo(
    artifact_size_bytes: int | None = None,
    parameter_size: str | None = None,
    quantization: str | None = None,
    est_ram_bytes: int | None = None,
    est_vram_bytes: int | None = None,
    observed_vram_bytes: int | None = None,
)

Facts about a local model artifact, used for tuning and recommendation.

Attributes:

Name	Type	Description
artifact_size_bytes	int | None	On-disk size of the model weights.
parameter_size	str | None	Parameter count as the runtime reports it (e.g. "7B").
quantization	str | None	Quantization scheme of the artifact (e.g. "Q4_K_M").
est_ram_bytes	int | None	Estimated system memory needed to run the model.
est_vram_bytes	int | None	Estimated GPU memory needed to run the model.
observed_vram_bytes	int | None	GPU memory actually measured in use while the model was loaded, when the runtime reports it.

anyinfer.ContextBudget dataclass

ContextBudget(
    context_window: Sourced[int] | None,
    estimate: RequestEstimate,
    output_reserve_tokens: int,
    headroom_tokens: int,
    pricing: Sourced[Pricing] | None = None,
)

A request's estimated size held against a model's known capacity.

The verdict is tri-state: fits is True/False when the context window is known, and None when it is not — an unknown capacity is reported as unknown, never guessed.

Attributes:

Name	Type	Description
context_window	Sourced[int] | None	The model's context window with its provenance, or None when nothing trustworthy is known.
estimate	RequestEstimate	The per-component input-token estimate.
output_reserve_tokens	int	Tokens reserved for the response.
headroom_tokens	int	Safety margin against estimation error.
pricing	Sourced[Pricing] | None	The model's per-token rates with their provenance, when known.

input_allowance_tokens property

input_allowance_tokens: int | None

Tokens the input may spend, or None when the window is unknown.

remaining_tokens property

remaining_tokens: int | None

Allowance left after the estimated input; negative when over budget.

This is the number an app packs context against: keep adding material while it stays positive.

fits property

fits: bool | None

Whether the estimated request fits the allowance; None when unknowable.

estimated_cost property

estimated_cost: CostEstimate | None

A preflight cost range, or None when no trustworthy pricing exists.

Estimated money never mixes with reported money: cost_usd is only ever computed from provider-reported usage, and this range is only ever computed from the estimate.

anyinfer.TokenEstimate dataclass

TokenEstimate(tokens: int, floor: int)

A token count carried as a planning estimate and a defensible lower bound.

Attributes:

Name	Type	Description
tokens	int	The planning figure, deliberately conservative-high.
floor	int	A lower bound the true count is not realistically below. An exact tokenizer sets floor == tokens.

__add__

__add__(other: TokenEstimate) -> TokenEstimate

Sum two estimates component-wise.

anyinfer.TokenEstimator

Bases: Protocol

Pluggable token counting.

Implementations may be heuristic (the shipped default) or exact (tiktoken, a provider's tokenize endpoint). Exact implementations should return TokenEstimate(n, n) so the gate can act on their counts with full force.

estimate

estimate(text: str) -> TokenEstimate

Estimate the token count of text.

anyinfer.HeuristicTokenEstimator dataclass

HeuristicTokenEstimator(multiplier: float = 1.0)

The dependency-free default: token counts from UTF-8 byte counts.

Attributes:

Name	Type	Description
multiplier	float	Calibration factor applied to the planning estimate, for providers whose transport envelope inflates reported prompt tokens beyond the serialized bytes. The floor is never inflated — envelope overhead is not something a lower bound may claim.

__post_init__

__post_init__() -> None

Reject non-finite or non-positive calibration factors.

estimate

estimate(text: str) -> TokenEstimate

Estimate tokens as ceil(bytes/3), with a bytes//8 floor.

anyinfer.RequestEstimate dataclass

RequestEstimate(
    messages: TokenEstimate,
    tools: TokenEstimate,
    schema: TokenEstimate,
)

Content-free size accounting for one request, by component.

The breakdown follows the typed request itself: what the caller said, what tools were offered, and what schema was attached — the three things that occupy input tokens on any provider.

Attributes:

Name	Type	Description
messages	TokenEstimate	The conversation, including per-message wire-framing overhead.
tools	TokenEstimate	Serialized tool specifications, when any were offered.
schema	TokenEstimate	The structured-output schema, when one was requested. Counted whether the wire carries it natively or the core injects it into the prompt — either way it occupies input tokens.

tokens property

tokens: int

Total planning estimate across all components.

floor property

floor: int

Total lower bound across all components.

anyinfer.build_context_budget

build_context_budget(
    request: GenerationRequest,
    capabilities: ModelCapabilities | None,
    *,
    estimator: TokenEstimator | None = None,
    output_reserve_tokens: int | None = None,
    headroom_tokens: int | None = None,
) -> ContextBudget

Compute the context budget for one request against one model's capabilities.

Parameters:

Name	Type	Description	Default
request	GenerationRequest	The request to size.	required
capabilities	ModelCapabilities | None	Assembled capabilities supplying the context window and maximum output size. None means nothing is known — the budget stays tri-state.	required
estimator	TokenEstimator | None	Token counting strategy; defaults to the byte heuristic.	None
output_reserve_tokens	int | None	Overrides the derived output reserve.	None
headroom_tokens	int | None	Overrides the default clamped headroom.	None

Returns:

Type	Description
ContextBudget	The computed ContextBudget.

Raises:

Type	Description
ValueError	If an explicit reserve or headroom is negative.

anyinfer.estimate_request

estimate_request(
    request: GenerationRequest,
    *,
    estimator: TokenEstimator | None = None,
) -> RequestEstimate

Estimate the input tokens a request will occupy.

Derived from the typed request rather than hand-fed strings: messages (every content part, plus per-message framing overhead), offered tools, and the schema.

Parameters:

Name	Type	Description	Default
request	GenerationRequest	The request to size.	required
estimator	TokenEstimator | None	Token counting strategy; defaults to the byte heuristic.	None

Returns:

Type	Description
RequestEstimate	The per-component estimate.

anyinfer.check_context_fit

check_context_fit(
    request: GenerationRequest,
    capabilities: ModelCapabilities | None,
    *,
    estimator: TokenEstimator | None = None,
    output_reserve_tokens: int | None = None,
    provider: str | None = None,
    model: str | None = None,
) -> ContextBudget

Build the budget for a request and raise if it provably cannot fit.

Parameters:

Name	Type	Description	Default
request	GenerationRequest	The GenerationRequest to size.	required
capabilities	ModelCapabilities | None	The target's assembled capabilities.	required
estimator	TokenEstimator | None	Token counting strategy; defaults to the byte heuristic.	None
output_reserve_tokens	int | None	Overrides the derived output reserve.	None
provider	str | None	Provider id, for the error's structured fields.	None
model	str | None	Model id, for the error message.	None

Returns:

Type	Description
ContextBudget	The computed ContextBudget when the
ContextBudget	request may proceed.

Raises:

Type	Description
ContextLengthError	When the estimate's floor exceeds a trusted-provenance context window.

anyinfer.CostEstimate dataclass

CostEstimate(
    low: Decimal, high: Decimal, currency: str = "USD"
)

A preflight cost range for one request.

Deliberately a range, never one number: the input estimate is two-sided (anyinfer.capabilities.estimate) and the output spend is unknown until the model stops. Kept strictly separate from cost_usd, which is only ever computed from reported usage — estimated and actual money must never be indistinguishable.

Attributes:

Name	Type	Description
low	Decimal	Floor input tokens priced, with zero output — the least this can cost.
high	Decimal	Planning-estimate input plus the full output reserve priced — a spend ceiling under the budget's own assumptions.
currency	str	The pricing currency.

anyinfer.PricingTable

PricingTable(entries: dict[str, tuple[PricingEntry, ...]])

Per-provider model pricing with prefix-aware lookup.

providers property

providers: tuple[str, ...]

Provider ids the table covers, sorted.

entries_for

entries_for(provider_id: str) -> tuple[PricingEntry, ...]

Every entry for one provider, or an empty tuple.

lookup

lookup(
    provider_id: str, model: str
) -> Sourced[Pricing] | None

Find pricing for a model: exact match first, then longest boundary prefix.

Returns:

Type	Description
Sourced[Pricing] | None	The pricing tagged catalog provenance, or None when the model has no
Sourced[Pricing] | None	entry — never a fallback price.

from_mapping classmethod

from_mapping(data: Any) -> PricingTable

Build and validate a table from parsed JSON.

Raises:

Type	Description
ConfigError	On a malformed document — wrong format version, missing fields, or prices that do not parse as non-negative decimals.

anyinfer.load_default_pricing cached

load_default_pricing() -> PricingTable

Load the pricing table bundled with this release.

anyinfer.fetch_pricing

fetch_pricing(
    url: str = DEFAULT_PRICING_URL,
    *,
    timeout_s: float = 30.0,
    transport: Any | None = None,
) -> PricingTable

Fetch a maintained pricing table over HTTPS — the explicit freshness opt-in.

Nothing in the library calls this implicitly. An application that wants prices newer than its installed release calls it on its own schedule and passes the result to the client's pricing_table.

Parameters:

Name	Type	Description	Default
url	str	Where to fetch from; defaults to the repo's continuously-updated file.	DEFAULT_PRICING_URL
timeout_s	float	Request timeout.	30.0
transport	Any | None	Test seam — an httpx2 transport.	None

Returns:

Type	Description
PricingTable	The fetched, validated table.

Raises:

Type	Description
ConfigError	If the fetch fails or the response is not a valid pricing document.