Skip to main content

​
What post-training does

A base model like Qwen3.5 arrives from pre-training fluent but generic. Post-training shapes it into one that is reliably good at your task, using your data and your definition of a good answer. You describe the task and pick a base model; Flash fine-tunes it on managed infrastructure and serves the result behind an OpenAI-compatible API.

​
The training loop

Every post-training run has the same three moving parts: a base model, an environment (your task and how it is graded), and a training algorithm that improves the model from a feedback signal. What differs is where that signal comes from.
1

Flash takes a prompt from your environment

Under SFT it also takes that row’s gold answer. Under GRPO and OPD the current model generates its own attempts at the prompt.
2

Something judges the result

SFT compares against the gold answer in your dataset. GRPO scores each attempt with your environment’s reward. OPD has a managed teacher grade your model’s tokens, so your reward is not the training signal.
3

The algorithm updates the model

The weights move toward whatever that signal rewards: the gold completions, the higher-scoring attempts, or the teacher’s distribution.
4

Repeat

Over many steps the model gets measurably better at the task. The output is a small adapter you can deploy.

​
Core concepts

​
Base models and LoRA adapters

Flash trains a LoRA adapter: a small set of extra weights layered on top of the frozen base model. This is parameter-efficient fine-tuning, with three practical payoffs:
  • Cheap and fast to train, because you’re updating a tiny fraction of the parameters.
  • Small to store and move (megabytes, not gigabytes).
  • Efficient to serve, because many adapters that share a base model can be served by the same managed service.
Pick the base model with one line in your config. Browse the catalog with flash models list, or see Supported models.

​
Environments: your task, as code

An environment is the task, expressed as code: a dataset of prompts your model practices on, and a reward that scores an answer. It is the single source of truth for what the model practices on and how it’s graded. You author one locally, publish it to the managed Environments Hub, and reference it from your config by Freesolo id. See the Environment model for the concepts, or Environments and Datasets to build one.

​
Three ways to teach: SFT, GRPO, and OPD

There are three ways to turn that environment into a better model, and you choose between them with one line of config.

SFT: learn by imitation

Supervised fine-tuning. You show the model good answers and it learns to reproduce them. Best when you already have examples of the behavior you want.

GRPO: learn by practice

Reinforcement learning. The model generates attempts, your reward scores them, and the model is pushed toward the higher-scoring ones. Best when good output is easier to score than to write out by hand.

OPD: learn from a teacher

On-policy distillation. The model generates attempts and a stronger teacher grades them token by token; the model is pulled toward the teacher. Best when a bigger model already does the task and you want a small one to match it — no answers to write, no reward to design.
Pick by what you can supply. See Training to configure one.

​
Rewards and rollouts (GRPO)

In GRPO, a rollout is one attempt the model generates for a prompt. For each prompt, GRPO samples a group of rollouts (the group_size), scores each one with your environment, and reinforces the rollouts that beat the group’s average. The reward is the score your environment returns. The reward is the teacher. If it reliably separates good answers from bad ones, GRPO can optimize toward it; if the task is so hard that every rollout scores zero, there’s no signal to learn from, so start with a model and task where some attempts succeed.

​
The teacher (OPD)

In OPD, a teacher — a managed model, GLM 5.2 by default or another you pick with teacher_model — grades your model’s own completions token by token, and training pulls your model toward the teacher’s distribution. Freesolo manages the teacher and its credentials, so there’s nothing to set up and you don’t pay for teacher tokens — only the GPU time the run uses. Because the signal only refines tokens your model already produces, OPD works best when you warm-start from an SFT adapter that already gets the format right; a cold OPD run tends to underperform plain SFT.

​
Serving the result

Deploy the adapter to call it. flash models deploy registers your adapter with Freesolo’s managed serving. You talk to it over an OpenAI-compatible API, and serving is billed per token. See Deploy & chat.

​
Is post-training right for your task?

It fits when the task is narrow and you can define success: you want a small, cheap model to do one job reliably instead of paying for a frontier model on every call, and prompting gets you close but not consistent. If you have neither data nor a way to grade answers, start there - the quality of your environment sets the ceiling on what training can achieve.

​
Next

Quickstart

Train, deploy, and chat with your first model in a few minutes.

Training in depth

SFT, GRPO, and OPD, config options, monitoring, and cost.