Registry, Catalog, and Credentials

How providers describe themselves (frozen descriptors, declarative setup specs), how targets and aliases resolve, and how credential references become secrets. Concepts: targets and aliases · credentials.

Provider Registry

anyinfer.ProviderRegistry

ProviderRegistry(
    *,
    load_builtins: bool = True,
    load_entry_points: bool = True,
)

Maps provider ids and aliases to descriptors, rejecting collisions.

Built-in providers are registered on first use. Entry-point providers are discovered lazily on the first lookup that misses, so installing a provider package is enough to make it resolvable.

register

register(
    descriptor: ProviderDescriptor, *, replace: bool = False
) -> None

Register a descriptor.

Parameters:

Name	Type	Description	Default
descriptor	ProviderDescriptor	The provider to register.	required
replace	bool	Allow replacing an existing registration of the same id. Aliases are still checked against other providers.	False

Raises:

Type	Description
ConfigError	On a duplicate id or an alias already claimed by another provider.

unregister

unregister(provider_id: str) -> None

Remove a provider and its aliases. Unknown ids are ignored.

resolve_alias

resolve_alias(name: str) -> str

Resolve a provider name or alias to a canonical provider id.

Raises:

Type	Description
ConfigError	If no provider claims the name.

get

get(provider_id: str) -> ProviderDescriptor

Look up a descriptor by id or alias.

Raises:

Type	Description
ConfigError	If no provider claims the name.

has

has(provider_id: str) -> bool

Whether a provider with this id or alias is registered.

known_ids

known_ids() -> tuple[str, ...]

Every registered canonical provider id, sorted.

__iter__

__iter__() -> Iterator[ProviderDescriptor]

Iterate descriptors in canonical-id order.

plugin_issues

plugin_issues() -> tuple[PluginLoadIssue, ...]

Third-party entry points that did not become usable providers.

Empty when every installed provider package loaded, which is the ordinary case. This call itself triggers discovery if it has not already run, so callers get a complete answer without touching the registry first.

anyinfer.ProviderDescriptor dataclass

ProviderDescriptor(
    id: str,
    display_name: str,
    factory: AdapterFactory,
    aliases: tuple[str, ...] = (),
    locality: Literal[
        "hosted", "local", "remote"
    ] = "hosted",
    default_base_url: str | None = None,
    requires_base_url: bool = False,
    honors_connection_settings: bool = True,
    setup: ProviderSetupSpec = ProviderSetupSpec(),
    reasoning_translator: ReasoningTranslator = _no_reasoning,
    static_capabilities: Mapping[
        str, ModelCapabilities
    ] = dict(),
    default_capabilities: ModelCapabilities = ModelCapabilities(),
    operations: frozenset[InferenceOperation] = frozenset(
        {"generation"}
    ),
    static_embedding_capabilities: Mapping[
        str, EmbeddingCapabilities
    ] = dict(),
    static_rerank_capabilities: Mapping[
        str, RerankCapabilities
    ] = dict(),
    token_calibration: TokenCalibration = TokenCalibration(),
    rate_limit_headers: RateLimitHeaders = RateLimitHeaders(),
    governs_own_transport: bool = False,
    supports_sessions: bool = False,
    model_puller: ModelPuller | None = None,
    model_inventory: Literal[
        "available", "installed", "served"
    ] = "served",
    uses_catalog: bool = False,
    reports_diagnostics: bool = False,
    cache_mechanism: CacheMechanism | None = None,
    cache_max_marks: int = 0,
    cache_min_tokens: int = 0,
    grammar_needs_prompt_injection: bool = False,
    max_repair_attempts: int | None = None,
    server_tools: frozenset[ServerToolKind] = frozenset(),
    ignored_parameters: tuple[str, ...] = (),
    derived_from: str | None = None,
)

Declarative facts about a provider and how to instantiate its adapter.

id instance-attribute

id: str

Canonical provider id — the provider half of a provider:model target. Normalized for lookup: lowercased, stripped, underscores to hyphens.

display_name instance-attribute

display_name: str

Human-readable name for UIs and error messages.

factory instance-attribute

factory: AdapterFactory

Builds this provider's adapter instance from its resolved configuration.

aliases class-attribute instance-attribute

aliases: tuple[str, ...] = ()

Alternative names that resolve to this provider; each must be globally unique across the registry.

locality class-attribute instance-attribute

locality: Literal['hosted', 'local', 'remote'] = 'hosted'

Where inference physically happens.

local means "on this machine" and carries two consequences — genuine zero pricing, and hardware detection that describes the right computer. remote is the third case a descriptor can never state on its own: an engine that is normally local, reached over a network. A client downgrades local to remote when the configured base URL is not loopback, because stamping zero cost on someone else's metered proxy, or sizing models against the wrong machine's RAM, are both silent wrong answers.

default_base_url class-attribute instance-attribute

default_base_url: str | None = None

Endpoint used when settings supply no base URL; None when there is no sensible default, as with per-tenant or supervised endpoints.

requires_base_url class-attribute instance-attribute

requires_base_url: bool = False

Whether the adapter cannot be built without a configured base URL.

honors_connection_settings class-attribute instance-attribute

honors_connection_settings: bool = True

Whether the adapter applies the instance's proxy/verify/client_cert.

True for every adapter that opens its own HTTP client through anyinfer.providers.http.build_client, which is nearly all of them. False for an adapter that delegates transport to a vendor SDK it does not configure — the parser reads this to refuse the keys at load rather than accept them and silently ignore them, which is the same "reject noise" rule that already turns verify: true into an error. Declared rather than hardcoded so a new delegating adapter inherits the behavior by saying so.

setup class-attribute instance-attribute

setup: ProviderSetupSpec = ProviderSetupSpec()

Declarative description of the configuration this provider needs, which is what a config UI renders.

reasoning_translator class-attribute instance-attribute

reasoning_translator: ReasoningTranslator = _no_reasoning

Maps normalized reasoning effort onto this provider's wire fields. The default translates every effort to nothing, for providers without a reasoning control.

static_capabilities class-attribute instance-attribute

static_capabilities: Mapping[str, ModelCapabilities] = (
    field(default_factory=dict)
)

Per-model capabilities declared ahead of time, keyed by model id; layered over default_capabilities when capabilities are assembled.

default_capabilities class-attribute instance-attribute

default_capabilities: ModelCapabilities = (
    ModelCapabilities()
)

Capabilities assumed for any model without a more specific source.

operations class-attribute instance-attribute

operations: frozenset[InferenceOperation] = frozenset(
    {"generation"}
)

Which inference operations this provider's adapter implements.

Defaults to generation-only, which is every existing adapter's actual behavior — this field is purely additive. A provider declaring "embedding" must build an adapter satisfying anyinfer.providers.base.EmbedsText; the client checks this when the adapter is first constructed rather than trusting the declaration blindly.

static_embedding_capabilities class-attribute instance-attribute

static_embedding_capabilities: Mapping[
    str, EmbeddingCapabilities
] = field(default_factory=dict)

Per-model embedding capabilities declared ahead of time, keyed by model id.

Populated only for models the provider actually embeds with; empty for a generation-only provider.

static_rerank_capabilities class-attribute instance-attribute

static_rerank_capabilities: Mapping[
    str, RerankCapabilities
] = field(default_factory=dict)

Per-model rerank capabilities declared ahead of time, keyed by model id.

Populated only for models the provider actually reranks with; empty for a generation-only provider.

token_calibration class-attribute instance-attribute

token_calibration: TokenCalibration = TokenCalibration()

How much this provider's transport inflates the prompt it is billed for.

Declared here rather than measured per request because it is a property of the provider's envelope, not of any one call: a session API that wraps the caller's messages in its own harness charges that harness on every request. The default is the identity — the provider counts what it was sent, and only a provider with evidence of a systematic gap should declare otherwise.

rate_limit_headers class-attribute instance-attribute

rate_limit_headers: RateLimitHeaders = RateLimitHeaders()

Which response headers this provider reports its rate-limit state in.

Empty by default, which means client-side pacing for this provider can only honour the bounds its caller configured. Declaring a dialect is what lets pacing anticipate the provider's own window instead — and, like every other wire fact here, it belongs in the provider's contract snapshot with a verified date.

governs_own_transport class-attribute instance-attribute

governs_own_transport: bool = False

Whether this provider builds its own transport rather than taking the core's.

True for adapters that talk through a vendor SDK or an interactive session instead of an httpx2 client the core constructed. The core cannot wrap what it did not build, so such a provider gets concurrency pacing only, applied around the call, and reports a dropped parameter if the caller asked for header-driven pacing it cannot perform.

supports_sessions class-attribute instance-attribute

supports_sessions: bool = False

Whether the provider can keep state between requests — a session API, or keep-alive model residency, rather than treating every request as independent.

model_puller class-attribute instance-attribute

model_puller: ModelPuller | None = None

How this provider is told to make a model available, or None when it cannot be.

For engines that keep their own model store, registry, and downloader — Ollama — the useful operation is not download these weights but make yourself ready. The implementation lives in anyinfer.local.services and is merely pointed at from here, both because acquisition never belongs in an adapter and because a declared hook keeps "which providers can do this" answerable from the registry rather than from a chain of engine checks in the core.

Weights fetched this way land in the engine's store under the engine's own name. Nothing is written to AnyInfer's model store and nothing is indexed there, so locate_model() will not find them — they are not ours to find.

model_inventory class-attribute instance-attribute

model_inventory: Literal[
    "available", "installed", "served"
] = "served"

What list_models() means for model-management UIs.

available is a catalog of things that could be run, installed is a provider-owned on-disk store, and served is the set an already-running engine exposes. The distinction prevents an application from presenting every catalog entry as though it were already installed.

uses_catalog class-attribute instance-attribute

uses_catalog: bool = False

Whether the core should supply its active catalog to this adapter.

Declared here so catalog composition is a provider fact, not a provider-id branch in client construction. Supervised engines use it to resolve model references while ordinary protocol adapters leave it false.

reports_diagnostics class-attribute instance-attribute

reports_diagnostics: bool = False

Whether this provider's adapter implements SupportsDiagnostics.

Declared rather than probed for, so "which providers can tell me about their runtime" is answerable from the registry alone. The core only calls diagnostics() on a provider that advertises it here.

cache_mechanism class-attribute instance-attribute

cache_mechanism: CacheMechanism | None = None

How this provider's prompt cache is engaged, or None when it offers nothing.

explicit means the wire format accepts per-segment cache marks and the adapter knows how to spell one. implicit means the provider caches stable prefixes by itself, so there is nothing to send and the core's whole duty is to leave the prefix alone. Declared here rather than inferred, because "does this provider cache" is a protocol fact recorded in its contract snapshot, not something to probe for.

cache_max_marks class-attribute instance-attribute

cache_max_marks: int = 0

Most explicit cache marks this provider accepts per request; 0 when it takes none. Exceeding a provider's ceiling is an error on some APIs and silently ignored on others, so the core clamps to this and reports the clamp.

cache_min_tokens class-attribute instance-attribute

cache_min_tokens: int = 0

Smallest segment this provider will actually cache, in tokens.

Below its own floor a provider bills a cache write and then never serves a read from it, so a mark placed there costs money and saves none. 0 means the provider states no floor.

grammar_needs_prompt_injection class-attribute instance-attribute

grammar_needs_prompt_injection: bool = False

Whether grammar mode also requires the schema in the prompt.

True for engines that compile the schema to a decoding grammar without conditioning the model on it (llama.cpp, and Ollama's format): the grammar guarantees well-formed JSON but not meaningful JSON unless the model was told the shape.

max_repair_attempts class-attribute instance-attribute

max_repair_attempts: int | None = None

The most schema-repair round trips this provider may be asked for, or None for no provider-imposed ceiling.

The repair budget is the caller's to set, and for almost every provider it should stay that way. A few cannot honor it: a provider whose every request is slow, interactively authenticated, or metered per conversation turn makes a second repair attempt cost far more than the malformed answer is worth, and one that keeps server-side conversation state is unlikely to answer a re-ask differently anyway.

Such a provider says so here, and the core clamps to it — visibly, as a ParameterDropped event, since a budget quietly reduced from three to one is exactly the kind of degradation this library refuses to perform in silence.

server_tools class-attribute instance-attribute

server_tools: frozenset[ServerToolKind] = frozenset()

Provider-run tools this adapter knows how to ask for.

Empty for almost every provider, and that emptiness is load-bearing: a request naming a server tool an adapter cannot spell is refused by the core before dispatch rather than sent without it. This is a stronger claim than a capability flag and is checked differently — a Feature is a fact about a model that may be a guess, while this is a fact about our own code and is never in doubt.

The refusal is deliberate where a dropped sampling knob would not be. An answer produced without the web search the caller asked for is not a degraded answer; it is a different one, built from stale training data, and it arrives looking exactly like a good one.

ignored_parameters class-attribute instance-attribute

ignored_parameters: tuple[str, ...] = ()

Request parameters this target will not honor, whatever the caller sets.

Two situations produce the same caller-visible outcome and so share one list: a provider that accepts the parameter and silently discards it, and one whose protocol has no field for it at all, so the adapter never sends it. Both are distinct from "rejects with an error" and from "supported" — in either case the request succeeds, the parameter does nothing, and nothing says so unless we say it. The core reports each as a ParameterDropped event instead of letting it pass unnoticed.

derived_from class-attribute instance-attribute

derived_from: str | None = None

The engine this descriptor is an instance of, when it is one.

An application that configures two Azure tenants or two OpenAI-compatible endpoints gives each instance its own id (work-azure, ollama-local); each becomes a descriptor derived from the underlying engine's, differing only in identity. None means this descriptor is an engine, which is the ordinary case.

identifiers property

identifiers: tuple[str, ...]

The id plus every alias, all normalized.

anyinfer.ProviderSetupSpec dataclass

ProviderSetupSpec(
    fields: tuple[SetupField, ...] = (),
    model_selection: Literal[
        "discover-or-manual", "manual-only"
    ] = "discover-or-manual",
    host_shorthand: HostShorthand | None = None,
    any_of: tuple[tuple[str, ...], ...] = (),
    requirement_note: str = "",
)

Everything a config UI needs to configure a provider without knowing which it is.

fields class-attribute instance-attribute

fields: tuple[SetupField, ...] = ()

The provider's configurable fields, in the order a UI should present them.

model_selection class-attribute instance-attribute

model_selection: Literal[
    "discover-or-manual", "manual-only"
] = "discover-or-manual"

Whether a UI may offer models discovered from the endpoint, or must let the user type a model id because the provider cannot enumerate what it serves.

host_shorthand class-attribute instance-attribute

host_shorthand: HostShorthand | None = None

Expansion rule applied when a bare hostname is entered as the base URL, or None when only full URLs make sense for this provider.

any_of class-attribute instance-attribute

any_of: tuple[tuple[str, ...], ...] = ()

Groups of field keys of which at least one must be supplied.

Some providers accept a choice of credential rather than a fixed one — Anthropic takes either an API key or a claude.ai OAuth token. That is a constraint over a group, so no per-field required flag can express it: marking both required demands both, and marking neither required lets an unconfigured instance save cleanly. Each inner tuple names the keys in one such group.

requirement_note class-attribute instance-attribute

requirement_note: str = ''

One line explaining the spec's requirements, shown beneath the fields.

Carried here rather than assembled by the UI because only the provider knows why its any_of groups exist; a generated sentence would say what is required without ever saying what the alternatives mean.

essential_fields property

essential_fields: tuple[SetupField, ...]

The fields a UI should put in front of the user, in declared order.

Everything the provider cannot supply a sensible answer for: credentials, and the endpoints and identifiers that vary per account. This is the short list an application prompts for.

advanced_fields property

advanced_fields: tuple[SetupField, ...]

The fields that already have a standard value, in declared order.

Offer them — a base URL override is what makes a proxy or a mirror usable — but offer them folded away, since changing one is the rare case rather than the setup path.

__post_init__

__post_init__() -> None

Reject a spec that hides a field a user cannot skip.

Marking a field both mandatory and advanced asks a UI to do two contradictory things, and the resolution it usually picks — honor the disclosure — produces the one failure mode worth designing against: a save that refuses, naming a field that is not on screen. Caught here, at import time, so it is a provider-authoring error rather than a user's dead end.

unsatisfied_groups

unsatisfied_groups(
    values: Mapping[str, str],
) -> tuple[tuple[str, ...], ...]

Return the any_of groups no value satisfies.

Empty when every group has at least one non-blank value, which is the case a UI needs to allow a save.

label_for

label_for(key: str) -> str

The declared label for a field key, falling back to the key itself.

anyinfer.SetupField dataclass

SetupField(
    key: str,
    label: str,
    kind: SetupFieldKind,
    required: bool = False,
    help_text: str = "",
    placeholder: str = "",
    env_var: str = "",
    advanced: bool = False,
    default_value: str = "",
    choices: tuple[str, ...] = (),
)

One configurable field a provider needs, described declaratively.

key instance-attribute

key: str

Machine name the entered value is saved under in provider settings — a well-known key such as api_key or api_version, or a provider-specific options entry.

label instance-attribute

label: str

Human-readable name a UI shows for the field.

kind instance-attribute

kind: SetupFieldKind

Semantic role of the field, which tells a UI how to render and validate it.

required class-attribute instance-attribute

required: bool = False

Whether saving needs a non-blank value for this field on its own. Either-or alternatives between fields are expressed via ProviderSetupSpec.any_of instead.

help_text class-attribute instance-attribute

help_text: str = ''

Explanatory sentence shown alongside the field; empty when the label suffices.

placeholder class-attribute instance-attribute

placeholder: str = ''

Example value a UI shows in the empty editor.

Declared per field because the right example is provider knowledge: the environment variable an Anthropic key conventionally lives in is not the one an OpenAI key does, and a UI that guesses picks one provider's convention and is wrong for all the others. Empty means the UI falls back to whatever generic hint suits the kind.

env_var class-attribute instance-attribute

env_var: str = ''

Environment variable this field is conventionally supplied from, if any.

The machine-readable half of what placeholder says in prose. A placeholder reading "env://ANTHROPIC_API_KEY or a literal key" is a UI hint; parsing it back out to learn which variable to look for is guessing at free text. Declared here, "is this provider already usable on this machine?" becomes a lookup rather than a regex, which is what anyinfer.local.discovery and a config UI's "we found this in your environment" both need.

The bare variable name, never the env:// reference form — the scheme is the credential resolver's spelling, and storing it here would make every consumer strip it. Empty when the provider has no convention, which is the case for a generic OpenAI-compatible endpoint and for every provider whose credential is not an environment variable at all.

advanced class-attribute instance-attribute

advanced: bool = False

Whether this field has a standard value that is right for almost every user.

The split this expresses is prominence, not optionality. required already says "saving fails without a value"; plenty of fields are neither required nor worth showing — an Ollama base URL, an Anthropic API version, a Bedrock signing profile. Presented as equals they read as five questions where there is really one, and every consuming application then has to rediscover which is which from prose help text.

So the provider says it here: advanced fields are the ones a UI may fold behind a disclosure, leaving the fields a user genuinely has to answer in front of them. A required field is never advanced — hiding something that blocks saving is exactly the trap this exists to avoid, and ProviderSetupSpec rejects that combination.

default_value class-attribute instance-attribute

default_value: str = ''

The value the provider falls back to when this field is left blank.

What a UI shows as "standard: …" beside a hidden field, so folding one away never hides what it will do. Empty when the field has no default — a credential has none, and neither does an endpoint the user must supply.

A UI should render this rather than pre-filling the editor with it: a saved copy of today's default is a value frozen at the moment someone opened a dialog, and it keeps overriding the real default long after that default has moved on.

choices class-attribute instance-attribute

choices: tuple[str, ...] = ()

The accepted values, for a choice field.

A bounded enum typed into a free-text box is a value that validates at request time rather than at configuration time, which turns a typo into a runtime error somewhere else entirely. Declaring the set here lets a UI offer it and lets a non-UI caller check a stored value without knowing which provider it belongs to.

Empty for every other kind — a field whose values a provider cannot enumerate has nothing to put here, and SetupField rejects the two contradictory combinations.

__post_init__

__post_init__() -> None

Reject a field whose declared choices and kind disagree.

A choice with no alternatives renders as an empty dropdown, and choices on a free-text field are a constraint no UI will apply. Both are provider-authoring errors, caught at import time rather than at the moment someone opens a dialog.

anyinfer.HostShorthand dataclass

HostShorthand(scheme: str, default_port: int)

Expansion rule for bare hostnames, e.g. myserver → http://myserver:11434.

scheme instance-attribute

scheme: str

URL scheme prepended when expanding a bare host, e.g. http.

default_port instance-attribute

default_port: int

Port appended when the bare host does not name one.

expand

expand(host: str) -> str

Expand a bare host into a full base URL, leaving full URLs untouched.

anyinfer.default_registry module-attribute

default_registry = ProviderRegistry()

The process-wide provider registry used when a client is given no other.

Deliberately not named registry: a module-level name equal to the module's own name shadows the module itself on the package, which breaks introspection, documentation generation, and patching in tests.

anyinfer.providers.builtin_descriptors

builtin_descriptors() -> Iterator[ProviderDescriptor]

Yield every built-in provider descriptor.

A module that fails to import — typically because its optional extra is absent — is skipped rather than breaking discovery for the others. The resulting "unknown provider" error, raised only if that provider is actually requested, carries the install hint.

Catalog

Two shapes over one body of data: the alias ladder ("just give me a good default") and the logical model table ("let me browse and pick"). Catalog.with_alias_target bridges them. See the model catalog.

anyinfer.Catalog dataclass

Catalog(
    aliases: Mapping[str, AliasEntry] = dict(),
    artifacts: Mapping[str, GgufArtifact] = dict(),
    models: Mapping[str, ModelEntry] = dict(),
    default_alias: str = "medium",
    format_version: int = FORMAT_VERSION,
)

A parsed alias catalog.

Attributes:

Name	Type	Description
aliases	Mapping[str, AliasEntry]	The tier ladder, keyed by lowercase alias name.
artifacts	Mapping[str, GgufArtifact]	Pinned GGUF artifacts, keyed by artifact id. Includes artifacts derived from the model table's GGUF variants as well as explicitly declared ones.
models	Mapping[str, ModelEntry]	The logical model table, keyed by model id.
default_alias	str	The alias resolution falls back to when a caller does not pick one.
format_version	int	Schema version of the parsed document.

has_alias

has_alias(name: str) -> bool

Whether an alias exists (case-insensitively).

alias

alias(name: str) -> AliasEntry

Look up an alias.

Raises:

Type	Description
ConfigError	If the alias is unknown.

targets_for_alias

targets_for_alias(name: str) -> Mapping[str, TargetEntry]

Every provider realization of an alias.

artifact

artifact(artifact_id: str) -> GgufArtifact

Look up a GGUF artifact.

Raises:

Type	Description
ConfigError	If the artifact is unknown.

alias_names

alias_names() -> tuple[str, ...]

Every alias name, sorted.

model

model(model_id: str) -> ModelEntry

Look up a logical model.

Raises:

Type	Description
ConfigError	If the model is unknown.

models_for

models_for(
    provider_id: str | None = None,
    *,
    best_at: str | None = None,
    kind: ModelKind | None = None,
) -> tuple[ModelEntry, ...]

Logical models filtered by serving channel, category, and kind, id-ordered.

Parameters:

Name	Type	Description	Default
provider_id	str | None	Keep only models this channel can serve.	None
best_at	str | None	Keep only models carrying this category tag.	None
kind	ModelKind | None	Keep only rows of this kind. None — the default — keeps every kind, because a caller browsing "what can this machine run" wants the embedding models too; a caller filling a chat picker passes "generation" rather than relying on a narrowing it never asked for.	None

with_alias_target

with_alias_target(
    alias: str, provider_id: str, model_id: str
) -> Catalog

Point one alias's provider target at a catalog model.

This is the bridge between browsing and the tier ladder: an app implements "use my catalog pick as medium" in one call, and the result resolves through the ordinary alias machinery with no resolver changes.

Raises:

Type	Description
ConfigError	If the alias or model is unknown, or the model has no artifact for the named provider.

overlay

overlay(other: Catalog) -> Catalog

Merge other on top of this catalog, entry by entry.

Application entries win over bundled ones at the alias, artifact, and model level — an overridden alias replaces the bundled one wholesale rather than merging its target map, so an app can remove a provider from a tier it does not want used. The same wholesale rule applies per model id.

from_mapping classmethod

from_mapping(data: Mapping[str, Any]) -> Catalog

Parse a catalog document.

Raises:

Type	Description
ConfigError	On an unsupported format version or malformed entries.

from_files classmethod

from_files(*paths: Path) -> Catalog

Load several catalog documents and overlay them left to right.

The bundled catalog is split this way on purpose: default.json stays a small, human-editable alias policy file while models.json is machine-maintained data with its own refresh cadence.

Raises:

Type	Description
ConfigError	If any file is missing, malformed, or empty of catalogs.

anyinfer.ModelEntry dataclass

ModelEntry(
    id: str,
    kind: ModelKind = "generation",
    family: str = "",
    display_name: str = "",
    parameter_size: str | None = None,
    quantization: str | None = None,
    context_window: int | None = None,
    license: str = "",
    best_at: tuple[str, ...] = (),
    est_file_bytes: int | None = None,
    est_ram_bytes: int | None = None,
    est_vram_bytes: int | None = None,
    last_verified: str = "",
    source: str = "",
    variants: tuple[ModelVariant, ...] = (),
    ollama: OllamaChannel | None = None,
    embedding: EmbeddingCapabilities | None = None,
    description: str = "",
)

One logical model in the catalog.

Attributes:

Name	Type	Description
id	str	Stable catalog id ("qwen2.5-7b-instruct").
kind	ModelKind	"generation" (the default, and every entry written before embeddings existed) or "embedding". See ModelKind.
family	str	Model family, for grouping in a UI.
display_name	str	Human-facing name.
parameter_size	str | None	Parameter class ("7B"), keying the KV-cache cost table.
quantization	str | None	The default quantization the headline estimates assume.
context_window	int | None	Native context length, when known.
license	str	License id; gated against the download allowlist.
best_at	tuple[str, ...]	Categories from BEST_AT.
est_file_bytes	int | None	Download size of the default variant.
est_ram_bytes	int | None	Memory needed on the CPU-only path.
est_vram_bytes	int | None	Memory needed when fully offloaded.
last_verified	str	ISO date the entry was actually checked against upstream.
source	str	URL of the upstream repository or registry page.
variants	tuple[ModelVariant, ...]	The quantization ladder, best quality first is not assumed — sort by ModelVariant.quality_rank.
ollama	OllamaChannel | None	The Ollama channel, when the model is published there.
embedding	EmbeddingCapabilities | None	Vector facts on kind="embedding" rows only, as the same record the client reads — the catalog states only dimensions and max_input_tokens, the two an upstream model card actually publishes. Everything else about an embedding model is either provider-specific (batch ceilings) or measurable rather than declared (normalization, which probe_embedding() observes), so the catalog does not guess at it.
description	str	Free text for display.

name property

name: str

Display name, falling back to the id.

is_embedding property

is_embedding: bool

Whether this row describes an embedding model rather than a chat model.

channels property

channels: tuple[str, ...]

Provider ids that can serve this model, sorted.

gguf_artifact_id property

gguf_artifact_id: str | None

The artifact id of this model's headline GGUF variant, when there is one.

"Headline" means the quantization the entry's own memory estimates describe — the rung a browsing user is being shown; not the highest rung in the repository. Picking the largest would quietly hand someone a Q8_0 download after they read a Q4_K_M size.

variants_for

variants_for(
    engine: str | None = None,
) -> tuple[ModelVariant, ...]

Variants for one engine, best quality first.

variant

variant(variant_id: str) -> ModelVariant

Look up one variant.

Raises:

Type	Description
ConfigError	If the variant is unknown.

matches_best_at

matches_best_at(category: str | None) -> bool

Whether this entry carries a category tag (case-insensitively).

anyinfer.ModelVariant dataclass

ModelVariant(
    id: str,
    engine: str = "llama.cpp",
    kind: str = "gguf",
    quantization: str = "",
    quality_rank: int = 0,
    est_file_bytes: int | None = None,
    est_ram_bytes: int | None = None,
    est_vram_bytes: int | None = None,
    min_compute_capability: str | None = None,
    source: SourceRef = SourceRef(),
    artifact_id: str | None = None,
)

One (model, quantization, engine) rung of a model's ladder.

A variant carries its own source reference, because a quantized vLLM variant is usually a different repository while a quantized GGUF variant is usually a different file in the same repository. One schema covers both only because the reference is per variant rather than per model.

Attributes:

Name	Type	Description
id	str	Stable variant id, unique within the catalog.
engine	str	"llama.cpp" or "vllm".
kind	str	"gguf" or "hf_repo".
quantization	str	The quantization this rung ships ("Q4_K_M", "awq").
quality_rank	int	Ladder position; higher is better quality.
est_file_bytes	int | None	On-disk size of the weights.
est_ram_bytes	int | None	Memory needed on the CPU-only path.
est_vram_bytes	int | None	Memory needed when fully offloaded.
min_compute_capability	str | None	NVIDIA compute capability this variant's kernels need, as a string ("8.9"). None means no gate.
source	SourceRef	Where the bytes come from.
artifact_id	str | None	For GGUF variants, the id under which the derived GgufArtifact is registered, so alias targets and the llama.cpp adapter can reference it.

is_pinned property

is_pinned: bool

Whether every declared file carries a revision and a digest.

anyinfer.OllamaChannel dataclass

OllamaChannel(tag: str, digest: str | None = None)

How a logical model is packaged in the Ollama registry.

We never download these — the daemon owns its own blob store. The tag is what we recommend, and digest is what drift checking compares against, because registry tags are mutable by design.

anyinfer.BEST_AT module-attribute

BEST_AT: frozenset[str] = frozenset(
    {
        "agentic",
        "code-completion",
        "coding",
        "drafting",
        "embeddings",
        "general-chat",
        "long-context",
        "low-resource",
        "math",
        "multilingual",
        "rag",
        "reasoning",
        "tool-use",
        "vision",
    }
)

The closed vocabulary of "best at" categories.

Closed on purpose: a free-text tag set drifts into synonyms nobody can filter on. Adding a category is a deliberate edit here, and the catalog validator enforces the set.

anyinfer.ModelKind module-attribute

ModelKind = str

What a catalog row is for: "generation" or "embedding".

The two share every acquisition mechanism — a GGUF is a GGUF, and the resolver, digests, and download machinery never care what the weights compute. What differs is interpretation, and it differs enough that guessing is wrong: an embedding model has no KV cache sized for a chat context, no quality ladder a user is trading off against throughput, and no place in the small/medium/large tier system, which answers "how big a chat model should I run". One field distinguishes them so the pieces that genuinely differ can ask, rather than a second table duplicating the pieces that do not.

anyinfer.MODEL_KINDS module-attribute

MODEL_KINDS: frozenset[str] = frozenset(
    {"generation", "embedding"}
)

The closed vocabulary of ModelEntry.kind, enforced by the parser and the validator.

anyinfer.load_default_catalog

load_default_catalog() -> Catalog

Load the catalog bundled with this AnyInfer build.

Two documents, overlaid: default.json carries the hand-edited alias policy, and models.json carries the machine-maintained logical model table with its own refresh cadence — the same split the bundled pricing table uses.

Browsing the Local Catalog

What Client.local_catalog() returns: every catalog model annotated with whether it fits, and why.

anyinfer.CatalogView dataclass

CatalogView(
    entries: tuple[CatalogEntryFit, ...] = (),
    hardware: HardwareProfile | None = None,
    hardware_source: HardwareSource = "unavailable",
    backend: Backend | None = None,
    notes: tuple[str, ...] = (),
)

A filtered, fit-annotated view of the local model catalog.

Attributes:

Name	Type	Description
entries	tuple[CatalogEntryFit, ...]	Models, best-fit-first.
hardware	HardwareProfile | None	The profile fits were judged against, when there was one.
hardware_source	HardwareSource	"detected" (probed this machine), "provided" (the caller supplied specs), or "unavailable" — the cue to collect a remote host's specs from the user and call again.
backend	Backend | None	The llama.cpp runtime variant that would actually drive these, when one is installed.
notes	tuple[str, ...]	View-level remarks, such as the runtime a machine should install.

runnable property

runnable: tuple[CatalogEntryFit, ...]

Only the entries this machine can plausibly run.

__len__

__len__() -> int

How many entries the view holds.

__iter__

__iter__() -> Any

Iterate the entries, best fit first.

anyinfer.CatalogEntryFit dataclass

CatalogEntryFit(
    model: ModelEntry,
    fit: ModelFit,
    channels: tuple[str, ...] = (),
)

One catalog model, judged against a machine.

Attributes:

Name	Type	Description
model	ModelEntry	The catalog entry.
fit	ModelFit	How it fits, with reasons.
channels	tuple[str, ...]	Provider ids that can serve it.

id property

id: str

The model id.

name property

name: str

The display name.

Credentials

anyinfer.CredentialResolver

Bases: Protocol

Resolves credential references of one scheme.

handles

handles(reference: str) -> bool

Whether this resolver recognizes reference.

resolve

resolve(reference: str) -> str

Resolve reference to a secret.

Raises:

Type	Description
CredentialError	If the reference is recognized but cannot be resolved.

anyinfer.ResolverChain

ResolverChain(
    resolvers: list[CredentialResolver],
    *,
    plugin_issues: Sequence[PluginLoadIssue] = (),
)

Tries each resolver in order, returning the first match's result.

The chain; not the individual resolvers — is responsible for registering resolved secrets for redaction, so a third-party resolver cannot forget to.

plugin_issues

plugin_issues() -> tuple[PluginLoadIssue, ...]

Entry points under anyinfer.credential_stores that did not become resolvers.

Mirrors ProviderRegistry.plugin_issues deliberately, and for the same reason: a skipped plugin is invisible at the point it matters, where the only symptom is that a reference nothing installed can resolve fails with a scheme error. The chain records rather than raises, so the failure of one vault plugin cannot stop a process whose other credentials resolve fine — but the record has to be reachable.

add

add(
    resolver: CredentialResolver, *, first: bool = True
) -> None

Register an additional resolver, by default ahead of the built-ins.

resolve

resolve(reference: str | None) -> str | None

Resolve a credential reference.

Parameters:

Name	Type	Description	Default
reference	str | None	The reference string, or None for "no credential configured".	required

Returns:

Type	Description
str | None	The resolved secret, or None when reference is None or empty.

Raises:

Type	Description
CredentialError	If no resolver handles the reference, or resolution fails.

anyinfer.default_resolver

default_resolver() -> ResolverChain

Build the standard resolver chain: plugins, keyring, env, then literal.

Literal is last because it accepts anything; the scheme-specific resolvers must get first refusal.

Resolvers published under the anyinfer.credential_stores entry-point group are placed ahead of the built-ins, so an organization's own vault scheme can be used from a plain config file — the sidecar has no other way to reach one. A plugin that claims a built-in scheme is dropped before it gets there, so being first in line cannot become a way to interpose on env:// or credential://; see anyinfer.plugins.load_credential_stores.

A plugin that fails to load is skipped rather than raising: an unavailable vault must not stop a process whose other credentials resolve fine. Each skip is warned once here and kept on the chain's ResolverChain.plugin_issues, because a silently skipped resolver is indistinguishable from a mistyped scheme at the point it fails.