Deploy & chat

Deploy a trained adapter and chat with it from the CLI or any OpenAI-compatible client, once its training run reaches done. flash deploy registers your adapter with Freesolo’s managed serving service. Send requests to it and pay per token (see Billing).

Deploy

flash deploy <run-id>

Preview what a deploy would do without creating it:

flash deploy <run-id> --dry-run

Every real deploy resolves the adapter to an immutable Hugging Face commit, runs a bounded serving smoke against that revision, and only then atomically activates the stable run-id alias. Verification is mandatory; a failed smoke leaves the existing alias unchanged.

Deploy a specific checkpoint

Every training step that saved an adapter is independently deployable. List a run’s deployable checkpoints:

flash checkpoints <run-id>

Then serve a specific step instead of the final adapter:

flash deploy <run-id>/step-<N>

Deploy a checkpoint while a run is still training, or after a GRPO run stops with useful intermediate steps. Checkpoint deployments attach serving metadata to the run without altering its training state.

A run that never finalized (cancelled or preempted mid-training) has no final adapter, so a plain flash deploy <run-id> fails with an error that lists the saved checkpoint steps and the exact flash deploy <run-id>/step-N command to deploy one of them instead.

Stable aliases and immutable revisions

Final adapters use RUN_ID@final.<40-char-sha> and saved checkpoints use RUN_ID@step-N.<40-char-sha>. The stable run-id alias moves only after the new revision passes its bounded smoke. Older revisions can remain directly callable. RUN_ID/step-N is a deploy and export selector, not a chat model name. After deploying that checkpoint, use the run alias or the full immutable revision from flash deployments.

Billing

Serving is billed per token. Prompt and completion tokens have per-model rates, and cached prompt tokens use the model’s cached-input rate. See prices in Supported models. Cached prompt tokens are cheaper than uncached prompt tokens. Prefix caching is always on: when a request’s prompt shares a leading prefix with a recent one — a shared system prompt, or the growing context of a multi-turn chat — the serving engine reuses that prefix from cache instead of recomputing it. Only the input prefix can be cached, so completion tokens always bill at the full per-token rate. When a request hits the prefix cache, the response reports how many prompt tokens were served from cache in usage.prompt_tokens_details.cached_tokens. Serving meters prompt, completion, and cached prompt tokens from the response and bills the org atomically. Sub-cent usage is carried forward, so very small requests are still accounted for without rounding each request up to a cent.

Chat from the CLI

flash chat <run-id> -m "Summarize the plot of Hamlet in two sentences."

flash chat accepts the stable run alias or a full immutable revision. RUN_ID/step-N is not accepted because it is a deploy/export selector. -m/--message is required; --system, --max-tokens (default 512), and --temperature (default 0.0) are optional. See the CLI reference for the full flag list.

flash chat <run-id> -m "Write a haiku about mountain weather" --temperature 0.8 --max-tokens 128

--system is transient — it applies to that one request and is not stored with the deployment. Use it to probe the adapter with the same system prompt it was trained with:

flash chat <run-id> -m "What is 6*7?" --system "Answer with just the number."

Manage deployments

flash deployments         # human-readable deployment status
flash deployments --json  # complete machine-readable records
flash undeploy <run-id>   # disable the alias and immutable revisions

Human output shows run id, step, revision, state, verification time, OpenAI model, and detail. JSON output includes openai_base_url for clients. flash undeploy <run-id> disables the stable alias and all of the run’s immutable revisions.

Use it from your own code

Deployments are OpenAI-compatible. Use flash deployments --json and pass its openai_base_url as the SDK base URL. Set model to the stable run alias or a full immutable revision. A valid Freesolo API key is required, and the key’s org must own the adapter named in model:

from openai import OpenAI

client = OpenAI(
    base_url="https://<your-endpoint>/v1",   # use openai_base_url from deployments json
    api_key="<your-freesolo-api-key>",       # use the same freesolo key as flash login
)

resp = client.chat.completions.create(
    model="<run-id>",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

For deployed adapters, serving preserves the run’s trained thinking mode. A caller cannot override enable_thinking per request for an adapter.

Structured outputs

Constrain a response to valid JSON with the OpenAI-standard response_format. A training structured_outputs constraint becomes the deployed adapter’s default. A request-level response_format can override it with text, json_object, or json_schema. This works on adapters and base models:

resp = client.chat.completions.create(
    model="<run-id>",
    messages=[{"role": "user", "content": "Give me a person as JSON."}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "schema": {
                "type": "object",
                "properties": {"name": {"type": "string"}, "age": {"type": "integer"}},
                "required": ["name", "age"],
            }
        },
    },
)

{"type": "json_object"} forces any valid JSON with no fixed schema, and {"type": "text"} leaves output unconstrained. With thinking enabled, reasoning is free-form and the grammar begins after </think>. Real deployments fail before alias activation if structured output is invalid, unclosed, or truncated. See the Structured outputs guide to also constrain training rollouts.

Chat with a base model (no adapter)

You don’t need a trained adapter to serve a catalog base model. Point the same OpenAI-compatible client at the serving endpoint and set model to a base-model id — any id from flash models, e.g. Qwen/Qwen3.5-4B — to generate against the base weights with no LoRA and no flash deploy:

resp = client.chat.completions.create(
    model="Qwen/Qwen3.5-4B",   # a base-model id, not a run id — no deploy needed
    messages=[{"role": "user", "content": "Hello!"}],
)

Any valid Freesolo API key reaches a base model — there is no adapter to own — and the tokens are billed to your own org at that model’s serving prices. Unlike a deployed adapter, which serves the reasoning behavior it was trained with, a base model honors the enable_thinking you pass in chat_template_kwargs.

resp = client.chat.completions.create(
    model="Qwen/Qwen3.5-4B",
    messages=[{"role": "user", "content": "Answer directly: 17 * 19"}],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)

Export to your own HuggingFace repo

Copy a trained adapter out of Freesolo’s managed storage into a HuggingFace repo you own:

flash export --adapter-id <run-id> --repository <owner>/<name>

--adapter-id and --repository are required; --api-key (defaults to HF_TOKEN) and --public are optional. See the CLI reference for the full flag list.

Get started

Flash

Reference

Deploy

Deploy a specific checkpoint

Stable aliases and immutable revisions

Billing

Chat from the CLI

Manage deployments

Use it from your own code

Structured outputs

Chat with a base model (no adapter)

Export to your own HuggingFace repo

Next steps

Serving prices

CLI reference

​Deploy

​Deploy a specific checkpoint

​Stable aliases and immutable revisions

​Billing

​Chat from the CLI

​Manage deployments

​Use it from your own code

​Structured outputs

​Chat with a base model (no adapter)

​Export to your own HuggingFace repo

​Next steps

Serving prices

CLI reference

Deploy

Deploy a specific checkpoint

Stable aliases and immutable revisions

Billing

Chat from the CLI

Manage deployments

Use it from your own code

Structured outputs

Chat with a base model (no adapter)

Export to your own HuggingFace repo

Next steps