Skip to content

LM Studio

LM Studio's local server. Generation uses its OpenAI-compatible endpoint, a shared and well-understood dialect; discovery uses the native API, which reports what a local engine's model list actually holds.

streaming structured output tool calls discovery (context, quantization, residency) health

Setup

import anyinfer as ai

client = ai.Client([ai.ProviderSettings.of("lm-studio")])

result = client.generate(prompt, target="lm-studio:qwen3-8b")

Defaults to http://127.0.0.1:1234/v1, LM Studio's conventional address. A bare hostname expands to that port, so a server on another machine needs only its name:

ai.ProviderSettings.of("lm-studio", base_url="gpu-box")  # http://gpu-box:1234

An API token is only needed when LM Studio's authentication has been enabled. lmstudio: is an accepted alias.

Discovery

The compatibility endpoint lists model ids. The native one lists what matters for a local engine:

for model in client.models("lm-studio"):
    caps = model.capabilities
    print(model.id, caps.context_window.value, caps.local.quantization)
    # qwen3-8b 32768 Q4_K_M

Context length, quantization, artifact size, tool-use and reasoning support, all with discovered provenance, because the server reported them rather than a table guessing. Embedding models are filtered out; they are not chat models.

Older LM Studio builds have no native API. A 404 there degrades to the OpenAI listing (ids alone) rather than failing.

Residency

On a local engine the difference between a fast request and a thirty-second wait is whether the model is already loaded. Health says so:

health = client.health("lm-studio")
print(health.detail)  # "loaded: qwen3-8b"  or  "no model loaded; the first request will load one"

The detail names every resident model, so an application that prefers an already-loaded model over one that would first be read from disk can route on it.

Reasoning

LM Studio names reasoning levels:

result = client.generate(prompt, target="lm-studio:qwen3-8b", reasoning="medium")

minimal maps to the server's low rather than off; disabling reasoning changes the answer more than reducing it does. Pass provider_options={"lm-studio": {"reasoning": "off"}} to turn it off deliberately.

Embeddings

Embedding models loaded in LM Studio serve through the OpenAI-compatible dialect:

result = client.embed(["hello"], target="lm-studio:text-embedding-nomic-embed-text-v1.5")

client.models("lm-studio", operation="embedding") lists which loaded models embed; the native listing's type field distinguishes them, so nothing is guessed. LM Studio documents no request limits for the endpoint; for large corpora set BatchPolicy.max_items_override to a size the machine handles.

Model Management

This adapter reads inventory but does not manage it: loading, unloading, and downloading stay in LM Studio's own UI and CLI (lms load, lms unload). Requests load a model on demand as usual.

For an engine AnyInfer supervises end to end (downloading artifacts, tuning for the local hardware, and managing the server process), see llama.cpp. For other engines, vLLM, SGLang, KoboldCpp, Jan, GPT4All, text-generation-webui, and TabbyAPI are all preconfigured presets, and any OpenAI-compatible server works through openai-compat.

Wire Contract

For the exact request/response fields this adapter depends on, see contracts/lm-studio.md.

See Also