Skip to content

Google Vertex AI

The same Gemini models as the AI Studio API, over the same protocol, with enterprise addressing and Google Cloud authentication. AnyInfer reuses the Gemini adapter's translation wholesale — Vertex changes where requests go and how they are signed, not what they look like.

streaming structured output tool calls reasoning discovery (no listing endpoint)

Setup

import anyinfer as ai

client = ai.Client([
    ai.ProviderSettings.of(
        "vertex",
        options={"project": "my-gcp-project", "location": "global"},
    ),
])

result = client.generate(prompt, target="vertex:gemini-2.5-flash")

project is required — it is part of the request path, not a header. location defaults to global; newer models are served only from the global endpoint, while a regional value (us-central1) selects that region's host.

vertex-ai: and google-vertex: are accepted aliases.

Authentication

Vertex takes an OAuth access token, not an API key, so the credential is acquired and refreshed rather than configured once. Three ways, in precedence order:

Install google-auth, then let its standard credential chain select metadata-server, workload-identity, or local gcloud credentials:

pip install google-auth

ai.ProviderSettings.of("vertex", options={"project": "my-project"})

AnyInfer uses google-auth when it is installed. The library is optional because it is not needed for the explicit-token or service-account paths.

pip install "anyinfer[vertex]"

ai.ProviderSettings.of("vertex", options={
    "project": "my-project",
    "credentials_file": "/secrets/sa.json",
})

Falls back to GOOGLE_APPLICATION_CREDENTIALS. The JWT is signed and exchanged in-house, so this works without google-auth — though signing needs an RSA implementation, and the error says so if none is available.

ai.ProviderSettings.of("vertex", api_key="env://GCP_ACCESS_TOKEN",
                        options={"project": "my-project"})

From gcloud auth print-access-token. Used verbatim and never refreshed — its lifetime is yours to manage. Note this is an access token, not a Gemini API key; the two are not interchangeable.

Acquired tokens are cached until two minutes before expiry, so a long-running client pays for one exchange per hour rather than one per request.

Everything else is Gemini

Thinking levels, response schemas, function calling, and usage accounting all behave exactly as on the Gemini page — including reporting output_tokens as answer plus thinking, since both bill at the output rate:

result = client.generate(prompt, target="vertex:gemini-2.5-pro", reasoning="high")
print(result.usage.reasoning_tokens)

Discovery reports nothing, deliberately

Vertex exposes no listing endpoint comparable to AI Studio's. client.models("vertex") returns an empty list rather than a hardcoded table — an invented inventory presented as discovery is exactly what the provenance rules exist to prevent. Name models explicitly in the target, and supply capability_overrides if you want their windows known.

Health checks that a token can be acquired, without spending a generation.

Claude on Vertex

Vertex also serves Anthropic models, but through a different surface (rawPredict/streamRawPredict with the Messages body). This adapter does not cover it — point the Anthropic adapter at that endpoint instead.

See also