Telemetry and Redaction

Typed in-process events, payload-free by default, plus the redaction registry that keeps secrets out of everything. Concepts: telemetry · credentials and redaction.

Observing

anyinfer.Observer

Bases: Protocol

A telemetry sink.

Implementations receive every event the client emits. Keep on_event fast and non-blocking; queue work elsewhere if it might be slow.

on_event

on_event(event: TelemetryEvent) -> None

Handle one telemetry event.

anyinfer.TelemetryEvent module-attribute

TelemetryEvent = (
    RequestStarted
    | ArenaCompleted
    | TargetResolved
    | AttemptStarted
    | FirstToken
    | AttemptCompleted
    | RetryScheduled
    | FallbackTriggered
    | RepairAttempted
    | RequestCompleted
    | RequestFailed
    | ParameterDropped
    | UsageEstimated
    | ServerLifecycle
    | DownloadProgress
    | ContextReduced
    | ProviderDiagnostic
    | CachePlanned
    | RateLimitWaited
    | RateLimitObserved
    | CredentialRotated
    | BatchSubmitted
    | BatchCompleted
)

Any event an observer may receive.

Ready-Made Sinks

Two sinks for the common cases, so a structured record does not have to be hand-written. Both are content-free unless the subscription opts into payloads, and every string they emit is redacted. For spans and metrics a backend can aggregate, use the OpenTelemetry bridge below instead.

anyinfer.LoggingObserver

LoggingObserver(
    logger: Logger | str | None = None,
    *,
    level: int | str = logging.INFO,
)

Emit each event as a structured record on a stdlib logging.Logger.

The message is the event name; the full mapping is attached as an anyinfer_event record attribute, so a JSON log formatter can render it while a plain formatter still prints something readable.

Both arguments accept the string forms a configuration file can express, because observers blocks in anyinfer.toml reach this constructor through build_observers and can only carry JSON/TOML scalars. A bad level is rejected here, at construction, so build_observers' promise that a typo fails at load rather than at the first event holds for the option values as well as the observer name — an unresolvable level would otherwise raise inside isEnabledFor on every single event, which the dispatcher suppresses after one warning, leaving a silently empty log.

Parameters:

Name	Type	Description	Default
logger	Logger | str | None	Where to log, as a Logger or a logger name to resolve. Defaults to a logger named DEFAULT_LOGGER_NAME.	None
level	int | str	Level to log every event at, as an int or a level name such as "INFO" (case-insensitive). One level for all of them on purpose — severity is the application's judgement, and a library that logs some events at WARNING decides that for you.	INFO

Raises:

Type	Description
ValueError	If level is a string that names no known logging level. build_observers maps this to ConfigError for the config path.

Example

import logging logging.basicConfig(level=logging.INFO) observer = LoggingObserver() client.subscribe(observer) # doctest: +SKIP

on_event

on_event(event: TelemetryEvent) -> None

Log one event. Never raises.

anyinfer.JsonlObserver

JsonlObserver(path: str | Path, *, flush: bool = True)

Append each event to a file as one JSON object per line.

Opened once and held open, because reopening per event turns a telemetry sink into the slowest thing on the request path. Writes are locked: the sync facade dispatches from its background loop thread while the caller's own thread may be subscribing.

Parameters:

Name	Type	Description	Default
path	str | Path	File to append to. Parent directories are created. Created at mode 0600 on POSIX — even payload-free telemetry names targets, models, and spend. Windows has no equivalent through chmod, so the file is not owner-restricted there; put it somewhere whose ACL already excludes other accounts.	required
flush	bool	Flush after every line. The default (True) costs a syscall per event and is what makes the file useful for tail -f and for a crash post-mortem, which is most of why this exists. Set False for high-volume runs where the file is only read afterwards.	True

Example

with JsonlObserver("telemetry.jsonl") as observer: # doctest: +SKIP ... client.subscribe(observer)

on_event

on_event(event: TelemetryEvent) -> None

Append one event. Never raises for a serialization problem.

close

close() -> None

Close the file. Further events are dropped rather than raising.

__enter__

__enter__() -> JsonlObserver

Enter a context that closes the file on exit.

__exit__

__exit__(*exc: object) -> None

Close the file.

anyinfer.events.sinks.event_to_dict

event_to_dict(event: TelemetryEvent) -> dict[str, Any]

Render one event as a JSON-safe mapping, redacted.

The event's type name lands under event; every field keeps its own name. Shared by both sinks, and useful on its own for a caller writing a third one.

Request Lifecycle Events

anyinfer.RequestStarted dataclass

RequestStarted(
    request_id: str,
    targets: tuple[Target, ...],
    metadata: Mapping[str, str] = dict(),
    prompt_text: str | None = None,
    operation: InferenceOperation = "generation",
)

An inference request entered the router.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
targets	tuple[Target, ...]	The fallback chain as requested, in the order the router will try it.
metadata	Mapping[str, str]	Caller-supplied labels from the request, passed through untouched.
prompt_text	str | None	The prompt text; None unless the receiving observer registered with payloads=True. Generation only — embedding inputs and rerank documents are never carried on events.
operation	InferenceOperation	Which inference operation this request is. Defaults to "generation" so pre-existing consumers observe no change; the embed and rerank dispatchers stamp their own value.

anyinfer.TargetResolved dataclass

TargetResolved(request_id: str, target: ResolvedTarget)

A target string resolved to a concrete provider and model.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
target	ResolvedTarget	The concrete provider and model the target string resolved to.

anyinfer.AttemptStarted dataclass

AttemptStarted(
    request_id: str,
    target: ResolvedTarget,
    attempt_number: int,
)

An attempt against one resolved target began.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
target	ResolvedTarget	The resolved target being attempted.
attempt_number	int	1-based count of attempts against this target; retries increment it, and falling back to a new target restarts it at 1.

anyinfer.FirstToken dataclass

FirstToken(
    request_id: str, target: ResolvedTarget, at_ms: float
)

The first content delta arrived — the centrally-measured TTFT.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
target	ResolvedTarget	The resolved target that produced the token.
at_ms	float	Milliseconds from attempt start to the first content delta.

anyinfer.AttemptCompleted dataclass

AttemptCompleted(
    request_id: str,
    target: ResolvedTarget,
    usage: Usage,
    timing: Timing,
    finish_reason: str,
)

An attempt finished successfully.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
target	ResolvedTarget	The resolved target that served the attempt.
usage	Usage	Token usage for the attempt, merged across every report the provider sent.
timing	Timing	Centrally-measured attempt timings, comparable across providers.
finish_reason	str	Normalized reason generation stopped, e.g. stop or length.

anyinfer.RetryScheduled dataclass

RetryScheduled(
    request_id: str,
    target: ResolvedTarget,
    attempt_number: int,
    delay_s: float,
    error: ErrorInfo,
)

A retryable failure will be retried against the same target after a delay.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
target	ResolvedTarget	The resolved target that will be retried.
attempt_number	int	The 1-based attempt that just failed; the retry is the next one.
delay_s	float	Seconds the router sleeps before the retry.
error	ErrorInfo	Snapshot of the retryable failure.

anyinfer.FallbackTriggered dataclass

FallbackTriggered(
    request_id: str,
    from_target: ResolvedTarget,
    to_target: Target,
    error: ErrorInfo | None = None,
)

A target was abandoned; the router advanced to the next in the chain.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
from_target	ResolvedTarget	The resolved target that was abandoned.
to_target	Target	The next target string the router will try, not yet resolved.
error	ErrorInfo | None	Snapshot of the failure that caused the abandonment, or None when the switch was a content-policy redirect rather than an error.

anyinfer.RepairAttempted dataclass

RepairAttempted(
    request_id: str,
    target: ResolvedTarget,
    attempt_number: int,
    mechanism: Mechanism | None,
    errors: tuple[str, ...] = (),
    raw_text: str | None = None,
)

A schema violation triggered a repair re-prompt.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
target	ResolvedTarget	The resolved target being re-prompted.
attempt_number	int	1-based count of repair attempts within this generation.
mechanism	Mechanism | None	The structured-output mechanism in force when validation failed, when one was chosen.
errors	tuple[str, ...]	The schema-validation messages that triggered the repair.
raw_text	str | None	The response text that failed validation; None unless the receiving observer registered with payloads=True.

anyinfer.ParameterDropped dataclass

ParameterDropped(
    request_id: str,
    target: ResolvedTarget,
    parameter: str,
    reason: str,
)

A requested parameter was not honored as asked, because the target cannot.

Dropping a parameter silently is how a caller ends up debugging why temperature=0 had no effect. Every drop is observable instead. The same applies to a parameter honored only in part — a repair budget clamped to a provider's ceiling is reported here too, since a budget quietly reduced from three to one is no more discoverable than one ignored outright.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
target	ResolvedTarget	The resolved target the parameter was withheld from.
parameter	str	Name of the request parameter that was not honored, dotted for a field of a compound one (repair.max_attempts).
reason	str	Human-readable explanation of what the target did instead.

anyinfer.ProviderDiagnostic dataclass

ProviderDiagnostic(
    target: ResolvedTarget | None,
    diagnostic: Diagnostic,
    request_id: str | None = None,
)

A provider reported something about its own runtime.

Emitted after an attempt for providers that declare reports_diagnostics, and whenever diagnostics() is called directly. The same text also lands on Generation.warnings, so a caller reading only results still sees it; this event is for observers that want it correlated with the request that hit it.

Attributes:

Name	Type	Description
target	ResolvedTarget | None	The resolved target whose runtime is being described, or None when the diagnostic was collected outside a request.
diagnostic	Diagnostic	What the provider reported.
request_id	str | None	Correlation id, or None when collected outside a request — in which case the OTel bridge records it as a standalone span.

anyinfer.ContextReduced dataclass

ContextReduced(
    strategy: str,
    representation: str,
    candidate_count: int,
    selected_count: int,
    omitted_count: int,
    estimated_tokens: int,
    max_tokens: int,
    binding_constraints: tuple[str, ...] = (),
    calls: int = 0,
)

Context was reduced to fit a budget.

Emitted for a reduced document corpus and for a compacted message history alike — both emulate a larger context window, and emulation is observable rather than silent. Content-free by construction: counts and ceilings only, never paths, document text, or message text — a path name can itself be sensitive.

Attributes:

Name	Type	Description
strategy	str	The strategy requested (auto stays auto), or history for a compacted conversation.
representation	str	The strategy actually applied.
candidate_count	int	Documents, or messages — offered to the reducer.
selected_count	int	Documents represented at detail fidelity, or messages kept.
omitted_count	int	Documents not represented in detail, or messages dropped.
estimated_tokens	int	Planning-side estimate of the rendered envelope, or of the compacted conversation.
max_tokens	int	The budget the reduction was held to.
binding_constraints	tuple[str, ...]	Which ceilings excluded at least one document.
calls	int	Generation calls spent; non-zero only for distill.

anyinfer.ArenaCompleted dataclass

ArenaCompleted(
    request_id: str,
    target_count: int,
    strategy: str,
    agreement: int | None,
    calls: int,
    memoized_tool_calls: int,
    synthesized: bool,
)

A bounded multi-target arena finished, without carrying answer content.

anyinfer.CachePlanned dataclass

CachePlanned(
    request_id: str,
    target: ResolvedTarget,
    mechanism: str,
    mark_count: int,
    estimated_cacheable_tokens: int,
)

The core decided how to engage a target's prompt cache.

Emitted only when a policy was in force and the target offered a mechanism. A policy that found nothing to use reports a ParameterDropped instead, because the interesting fact there is the degradation, not the plan.

Content-free: counts and mechanism only. What was marked is a position, never text.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
target	ResolvedTarget	The resolved target the plan applies to.
mechanism	str	explicit for per-segment marks, implicit for prefix stability.
mark_count	int	How many marks were placed; always zero for implicit.
estimated_cacheable_tokens	int	Planning-side size of what the plan tries to cache. An intention, not a saving — realized savings come only from the provider's own reported usage.

anyinfer.RateLimitWaited dataclass

RateLimitWaited(
    request_id: str,
    provider_id: str,
    waited_s: float,
    reason: Literal[
        "concurrency", "interval", "provider-headers"
    ],
    target: ResolvedTarget | None = None,
)

A request was held back by client-side pacing before it was dispatched.

Emitted so a paced request never looks like a slow provider. The same wait also lands in the attempt's timing.phases["queued_ms"], because latency a caller cannot attribute is a support ticket.

Attributes:

Name	Type	Description
request_id	str	Correlation id of the waiting request; empty when the wait happened outside a tracked generation, as on a model listing.
provider_id	str	The provider instance whose limiter did the waiting.
waited_s	float	How long the request was held.
reason	Literal['concurrency', 'interval', 'provider-headers']	concurrency for an in-flight bound, interval for a configured rate or minimum gap, provider-headers for a window the provider itself reported.
target	ResolvedTarget | None	The resolved target, when the wait belongs to a generation attempt.

anyinfer.RateLimitObserved dataclass

RateLimitObserved(
    provider_id: str,
    requests_remaining: int | None = None,
    tokens_remaining: int | None = None,
    resets_in_s: float | None = None,
)

A provider reported its rate-limit state on a response.

Emitted at most once per response, and only when the provider declared a header dialect and actually populated it. Purely what the provider said — this library adds no estimate of its own, and a field the provider left out stays None rather than becoming a guess.

Attributes:

Name	Type	Description
provider_id	str	The provider instance that reported.
requests_remaining	int | None	Requests left in the current window, when stated.
tokens_remaining	int | None	Tokens left in the current window, when stated.
resets_in_s	float | None	Seconds until the window resets, when stated in a form that can be read as a duration.

anyinfer.UsageEstimated dataclass

UsageEstimated(
    request_id: str,
    target: ResolvedTarget,
    field_name: str,
    method: str,
)

A usage figure was derived rather than reported by the provider.

Estimated and reported numbers must never be indistinguishable downstream; this event is what marks the difference for observers.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
target	ResolvedTarget	The resolved target the estimate applies to.
field_name	str	The usage field that was estimated, e.g. input_tokens.
method	str	How the estimate was derived.

anyinfer.RequestCompleted dataclass

RequestCompleted(
    request_id: str,
    target: ResolvedTarget,
    usage: Usage,
    timing: Timing,
    repair_attempts: int = 0,
    response_text: str | None = None,
)

A request produced a result.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
target	ResolvedTarget	The resolved target that produced the result.
usage	Usage	Final token usage for the request.
timing	Timing	Final centrally-measured timings for the request.
repair_attempts	int	How many repair re-prompts were needed; 0 means the first response validated.
response_text	str | None	The final response text; None unless the receiving observer registered with payloads=True.

anyinfer.RequestFailed dataclass

RequestFailed(request_id: str, error: ErrorInfo)

A request exhausted its route without a result.

Attributes:

Name	Type	Description
request_id	str	Correlation id shared by every event this request emits.
error	ErrorInfo	Snapshot of the terminal failure, after every target and retry was spent.

Local Subsystem Events

anyinfer.DownloadProgress dataclass

DownloadProgress(
    artifact_id: str,
    downloaded_bytes: int,
    total_bytes: int | None,
    done: bool = False,
    phase: str = "",
    file_index: int = 0,
    file_count: int = 0,
    filename: str = "",
    session_bytes: int = 0,
)

Progress of a model acquisition.

downloaded_bytes and total_bytes are aggregate across the whole acquisition, which is what their names have always implied. Earlier builds reported them per file, so a sharded artifact restarted the counter at zero on every shard with no way for an observer to tell. The remaining fields carry the per-file detail that the aggregate figures deliberately no longer mix in.

Attributes:

Name	Type	Description
artifact_id	str	The artifact or catalog variant being acquired.
downloaded_bytes	int	Bytes present across every file, including bytes that were already on disk before this run, so resuming reports the resumed position.
total_bytes	int | None	Total expected bytes, or None when a size is genuinely unknown.
done	bool	Whether the acquisition finished.
phase	str	Which stage emitted this, when the acquisition engine supplied one.
file_index	int	1-based index of the file that most recently advanced.
file_count	int	How many files this acquisition covers.
filename	str	Name of that file.
session_bytes	int	Bytes this run actually transferred, as opposed to resumed.

anyinfer.ServerLifecycle dataclass

ServerLifecycle(
    server_id: str,
    state: Literal[
        "starting",
        "ready",
        "stopping",
        "stopped",
        "crashed",
    ],
    detail: str = "",
)

A supervised local server changed state.

Attributes:

Name	Type	Description
server_id	str	Identifies the supervised server — the key of the model it serves.
state	Literal['starting', 'ready', 'stopping', 'stopped', 'crashed']	The state the server just entered.
detail	str	Human-readable context, such as a stop reason or a tail of crash output; empty when there is nothing to add.

anyinfer.CredentialRotated dataclass

CredentialRotated(
    provider: str,
    trigger: Literal["ttl", "auth-failure"] = "ttl",
)

A provider instance's credential changed and its adapter was rebuilt.

Emitted only when the resolved value actually moved — a TTL expiring and the credential re-resolving to the same secret is not an event, because nothing happened. Rotation discards a connection pool mid-flight, so an operator watching latency deserves to know which second it was.

Payload-free by construction: the credential is never carried here, and neither is the change-detection digest.

Attributes:

Name	Type	Description
provider	str	The provider instance whose adapter was rebuilt.
trigger	Literal['ttl', 'auth-failure']	What prompted the re-resolution — "ttl" for the periodic re-check, "auth-failure" for the one triggered by a provider rejecting a credential that had been working.

anyinfer.BatchSubmitted dataclass

BatchSubmitted(
    batch_id: str, target: ResolvedTarget, line_count: int
)

A deferred batch was accepted by a provider.

Payload-free: the line count is a number, and the batch id is the provider's own handle rather than anything derived from the requests. Nothing about what was asked is carried, which is the same rule every other event here follows.

Attributes:

Name	Type	Description
batch_id	str	The provider's id for the job.
target	ResolvedTarget	Where it was submitted.
line_count	int	How many requests went in.

anyinfer.BatchCompleted dataclass

BatchCompleted(
    batch_id: str,
    target: ResolvedTarget,
    status: str,
    completed: int,
    failed: int,
)

A finished batch was collected.

Emitted on collection, not completion — nothing here polls, and a batch that finished overnight becomes observable the moment a caller asks for it.

Attributes:

Name	Type	Description
batch_id	str	The provider's id for the job.
target	ResolvedTarget	Where it ran.
status	str	The batch's terminal status.
completed	int	Lines that produced a result.
failed	int	Lines that did not.

Redaction

anyinfer.RedactionRegistry

RedactionRegistry()

A thread-safe set of secrets to strip from outbound strings.

register

register(secret: str | None) -> None

Register a secret for redaction, along with its common encodings.

Redaction is exact-substring matching, so a secret that reaches a string in an encoded form — JSON-escaped inside a serialized body, percent-encoded in a URL, base64'd in an auth header — would not match the raw value and would survive. Each such form is registered alongside the original, which is cheap (a handful of extra strings per credential) and closes the gap for the encodings that actually occur on the request path.

This is defense in depth, not a guarantee: an encoding nobody anticipated still slips through, which is exactly why the cassette audit exists as a second net.

Values shorter than MIN_SECRET_LEN are ignored: redacting them would corrupt unrelated text far more often than it would protect anything. The same floor is applied to each derived form.

redact

redact(text: str) -> str

Replace every registered secret in text with REDACTED.

clear

clear() -> None

Forget all registered secrets. Intended for tests.

__len__

__len__() -> int

Number of registered secrets — not the number of strings matched against.

One registration also stores that secret's encoded forms; counting those would make this number an implementation detail of the encoding list.

anyinfer.register_secret

register_secret(secret: str | None) -> None

Register a secret with the process-wide registry.

anyinfer.redact

redact(text: str) -> str

Redact registered secrets from text using the process-wide registry.

OpenTelemetry Export

The optional [otel] extra maps these events onto OpenTelemetry spans and metrics. Guide: observability.

anyinfer.otel.install

install(
    client: Any, *, record_payloads: bool = False
) -> OTelObserver

Attach an OTelObserver to a client.

Parameters:

Name	Type	Description	Default
client	Any	An AsyncClient or Client.	required
record_payloads	bool	Attach prompt and response text to spans.	False

Returns:

Type	Description
OTelObserver	The observer, so it can be detached later.

anyinfer.otel.OTelObserver

OTelObserver(
    *,
    tracer: Any = None,
    meter: Any = None,
    record_payloads: bool = False,
)

Maps AnyInfer telemetry events onto OpenTelemetry spans and metrics.

One span per request, with attempts as span events. Requests and attempts are correlated by request_id, so a fallback chain reads as a single trace rather than several disconnected ones.

Parameters:

Name	Type	Description	Default
tracer	Any	An OpenTelemetry tracer. Defaults to one from the global provider.	None
meter	Any	An OpenTelemetry meter. Defaults to one from the global provider.	None
record_payloads	bool	Attach prompt and response text to spans. Off by default, matching the payload-free default of the event contract itself. Subscribe with payloads=True as well for this to have any effect.	False

Raises:

Type	Description
ConfigError	If opentelemetry-api is not installed.

on_event

on_event(event: TelemetryEvent) -> None

Handle one telemetry event.

Never raises: the dispatcher isolates observer failures, but a telemetry bridge that can break a generation would be a poor trade regardless.

anyinfer.otel.GEN_AI module-attribute

GEN_AI = 'gen_ai'

Prefix of the GenAI semantic-convention attribute namespace.