Deploy & chat

Deploy a trained adapter and chat with it from the CLI or any OpenAI-compatible client, once its training run reaches done. flash models deploy registers your adapter with Freesolo’s managed serving service. Send requests to it and pay per token (see Billing).

One exception: a run whose config pinned model_revision cannot be deployed, and neither can an adapter warm-started from one. Serving does not support revision-pinned base models, so leave model_revision empty unless you need the pin and do not intend to deploy the result.

Deploy

flash models deploy <run-id>

Preview what a deploy would do without creating it:

flash models deploy <run-id> --dry-run

Every real deploy resolves the adapter to an immutable Hugging Face commit, runs a bounded serving smoke against that revision, and atomically activates the stable run-id alias only if the smoke passes; otherwise the existing alias is left unchanged. flash models deploy returns while the revision is still queued, so the model is not servable the moment the command exits. To gate a script on the deployment, add --wait:

flash models deploy <run-id> --wait

It blocks until the revision is servable and exits 0, or exits 1 if the deployment fails, rolls back, or the wait times out. The default timeout is 1800 seconds; pass --wait 600 for your own budget, or --wait 0 for a single state read that does not block. Interrupting with Ctrl-C stops the waiting, not the deployment. You can also deploy and undeploy from the model’s detail page in the dashboard. Both paths run the same verification and produce the same alias, so a deployment started in the web app is visible to flash models deployments and can be torn down with flash models undeploy.

Deploy a specific checkpoint

Every training step that saved an adapter is independently deployable. List a run’s deployable checkpoints:

flash runs checkpoint <run-id>

Then serve a specific step instead of the final adapter:

flash models deploy <run-id>/step-<N>

Deploy a checkpoint while a run is still training, or after a GRPO run stops with useful intermediate steps. Checkpoint deployments attach serving metadata to the run without altering its training state.

A run that never finalized (cancelled or preempted mid-training) has no final adapter, so a plain flash models deploy <run-id> fails with an error that lists the saved checkpoint steps and the exact flash models deploy <run-id>/step-N command to deploy one of them instead.

Stable aliases and immutable revisions

Final adapters use RUN_ID@final.<40-char-sha> and saved checkpoints use RUN_ID@step-N.<40-char-sha>. The stable run-id alias moves only after the new revision passes its bounded smoke. Older revisions can remain directly callable. RUN_ID/step-N is a deploy and export selector. Once you have deployed that checkpoint, flash models chat RUN_ID/step-N works from the CLI too, resolving to that checkpoint’s verified revision. From an OpenAI client, set model to the run alias or the full immutable revision from flash models deployments; the RUN_ID/step-N shorthand is a Flash-CLI selector, not an OpenAI model name.

Billing

Serving is billed per token. Prompt and completion tokens have per-model rates, and cached prompt tokens use the model’s cached-input rate. See prices in Supported models. Cached prompt tokens are cheaper than uncached prompt tokens. Prefix caching is always on: when a request’s prompt shares a leading prefix with a recent one — a shared system prompt, or the growing context of a multi-turn chat — the serving reuses that prefix from cache instead of recomputing it. Only the input prefix can be cached, so completion tokens always bill at the full per-token rate. When a request hits the prefix cache, the response reports how many prompt tokens were served from cache in usage.prompt_tokens_details.cached_tokens. Sub-cent usage is carried forward, so small requests are accounted for without rounding each one up to a cent.

Chat from the CLI

flash models chat <run-id> -m "Summarize the plot of Hamlet in two sentences."

flash models chat accepts the stable run alias, a full immutable revision, or RUN_ID/step-N for a checkpoint you have already deployed. A RUN_ID/step-N target resolves to that checkpoint’s verified revision, so deploy it first with flash models deploy RUN_ID/step-N. -m/--message is required; --system, --max-tokens (default 512), and --temperature (default 0.0) are optional. See the CLI reference for the full flag list.

flash models chat <run-id> -m "Write a haiku about mountain weather" --temperature 0.8 --max-tokens 128

--system is transient — it applies to that one request and is not stored with the deployment. Use it to probe the adapter with the same system prompt it was trained with:

flash models chat <run-id> -m "What is 6*7?" --system "Answer with just the number."

Manage deployments

flash models deployments         # human-readable deployment status
flash models deployments --json  # complete machine-readable records
flash models undeploy <run-id>   # disable the alias and immutable revisions

Human output shows run id, step, revision, state, verification time, OpenAI model, and detail. JSON output includes openai_base_url for clients.

Use it from your own code

Deployments are OpenAI-compatible. Use flash models deployments --json and pass its openai_base_url as the SDK base URL. Set model to the stable run alias or a full immutable revision. A valid Freesolo API key is required, and the key’s org must own the adapter named in model:

from openai import OpenAI

client = OpenAI(
    base_url="https://serve.freesolo.co/v1",  # openai_base_url from flash models deployments --json
    api_key="<your-freesolo-api-key>",       # use the same freesolo key as flash login
)

resp = client.chat.completions.create(
    model="<run-id>",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

For deployed adapters, serving preserves the run’s trained thinking mode. A caller cannot override enable_thinking per request for an adapter.

Reasoning is returned separately

When a generation runs in thinking mode, the reasoning comes back in reasoning_content and only the answer in content. The </think> separator is not included in either field:

resp = client.chat.completions.create(
    model="<run-id>",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.reasoning_content)  # how it got there
print(resp.choices[0].message.content)  # the answer

Streaming splits the same way: reasoning tokens arrive as choices[0].delta.reasoning_content and answer tokens as choices[0].delta.content, so accumulate both. If a generation stops before the reasoning closes, everything generated is returned as reasoning_content and content is empty. Non-thinking completions are unaffected and use content alone. flash models chat folds both fields into one balanced <think>...</think> block; only direct endpoint calls expose the split shape.

Send images

Every catalog model accepts image inputs, so a served adapter or base model can take multimodal prompts over the same OpenAI-compatible API. Pass images as OpenAI image_url content parts. Serving accepts base64 data: URIs only; remote (http(s)) and file: URLs are rejected. A single request accepts up to four images:

import base64

with open("photo.jpg", "rb") as f:
    data_uri = "data:image/jpeg;base64," + base64.b64encode(f.read()).decode()

resp = client.chat.completions.create(
    model="<run-id>",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is in this image?"},
                {"type": "image_url", "image_url": {"url": data_uri}},
            ],
        }
    ],
)

To train on image data, see Image inputs.

Structured outputs

Constrain a response to valid JSON with the OpenAI-standard response_format. A training structured_outputs constraint becomes the deployed adapter’s default. A request-level response_format can override it with text, json_object, or json_schema. This works on adapters and base models:

resp = client.chat.completions.create(
    model="<run-id>",
    messages=[{"role": "user", "content": "Give me a person as JSON."}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "schema": {
                "type": "object",
                "properties": {"name": {"type": "string"}, "age": {"type": "integer"}},
                "required": ["name", "age"],
            }
        },
    },
)

{"type": "json_object"} forces any valid JSON with no fixed schema, and {"type": "text"} leaves output unconstrained. With thinking enabled, reasoning is free-form and the grammar begins after </think>. Real deployments fail before alias activation if structured output is invalid, unclosed, or truncated. See the Structured outputs guide to also constrain training rollouts.

Chat with a base model (no adapter)

Using the same OpenAI-compatible client, set model to any id from flash models list, such as Qwen/Qwen3.5-4B, to generate against the base weights with no LoRA and no flash models deploy:

resp = client.chat.completions.create(
    model="Qwen/Qwen3.5-4B",   # a base-model id, not a run id — no deploy needed
    messages=[{"role": "user", "content": "Hello!"}],
)

Any valid Freesolo API key reaches a base model — there is no adapter to own — and the tokens are billed to your own org at that model’s serving prices. Unlike a deployed adapter, which serves the reasoning behavior it was trained with, a base model honors the enable_thinking you pass in chat_template_kwargs.

resp = client.chat.completions.create(
    model="Qwen/Qwen3.5-4B",
    messages=[{"role": "user", "content": "Answer directly: 17 * 19"}],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)

Export to your own HuggingFace repo

Copy a trained adapter out of Freesolo’s managed storage into a repo you own:

flash models export --adapter-id <run-id> --repository <owner>/<name>

--adapter-id and --repository are required; --api-key (defaults to HF_TOKEN) and --public are optional. See the CLI reference for the full flag list.

Get started

Flash

Reference

Deploy

Deploy a specific checkpoint

Stable aliases and immutable revisions

Billing

Chat from the CLI

Manage deployments

Use it from your own code

Reasoning is returned separately

Send images

Structured outputs

Chat with a base model (no adapter)

Export to your own HuggingFace repo

Next steps

Serving prices

CLI reference

​Deploy

​Deploy a specific checkpoint

​Stable aliases and immutable revisions

​Billing

​Chat from the CLI

​Manage deployments

​Use it from your own code

​Reasoning is returned separately

​Send images

​Structured outputs

​Chat with a base model (no adapter)

​Export to your own HuggingFace repo

​Next steps

Serving prices

CLI reference

Deploy

Deploy a specific checkpoint

Stable aliases and immutable revisions

Billing

Chat from the CLI

Manage deployments

Use it from your own code

Reasoning is returned separately

Send images

Structured outputs

Chat with a base model (no adapter)

Export to your own HuggingFace repo

Next steps