flash env setup scaffolds a starter project into the current directory. A
run is fully described by what lands on disk: an environment (your task and
how it’s scored) and a config (how to train on it). Every file is plain text
you can read, diff, and version-control. Rerunning is safe: existing files stay untouched.
environment.py
What it is: your environment, the single source of truth for what the model practices on and how it’s graded. It definesload_environment(), which returns a Freesolo EnvironmentSingleTurn (or
EnvironmentMultiTurn) carrying a dataset and a score_response reward. This
is the file you edit first.
When Flash reads it: flash env push packages and uploads it, and every real
training run imports it and calls load_environment(**params). SFT --cost,
--dry-run, and submit preparation do not import it: the control plane reads the
selected packaged JSON or JSONL dataset directly. GRPO and OPD --cost quote
offline from the catalog. The scaffolded starter loads its rows from
dataset/train.jsonl.
See the full scaffolded file — StarterEnv with build_prompt_messages
and score_response — in Environments.
evaluations.py
What it is: held-out evaluation suites for the environment, defined asBaseEvalSuite subclasses returning EvalCase rows. It is optional: delete it
and everything else still works. Keeping it gives you a fixed set of cases to
score a trained model against, separate from the reward the model trains on.
When Flash reads it: flash env eval scores
the suites against a deployed model, and flash env test validates them offline
without calling one. flash env push ships it alongside environment.py.
dataset/train.jsonl
What it is: a tiny starter dataset ofinput/output rows that the
scaffolded environment.py loads. Replace it with your training rows before a real run. See Datasets.
When Flash reads it: your environment.py reads it during real training.
For SFT estimates, the control plane reads the selected packaged JSON or JSONL
file directly and tokenizes raw input and output fields plus the static
training contract. It does not execute environment-defined prompt construction,
filtering, or transforms. flash env push uploads the dataset/ folder.
dataset/train.jsonl
configs/sft.toml
What it is: an SFT training config for supervised fine-tuning on theinput/output pairs in your environment’s
dataset. You set model, the [environment] id, and the [train] knobs
(epochs, lora_rank); the
training infrastructure and artifact storage are managed for you. Copy
it per experiment.
When Flash reads it: every flash train, --dry-run, and --cost parses
this file. Dry-run sends the spec to the authenticated server for submit-time
preflights. --cost stays local for GRPO and OPD. SFT --cost authenticates and returns a synchronous packaged-dataset estimate
without starting paid training or allocating a training GPU.
configs/sft.toml
configs/rl.toml and configs/opd.toml
What they are: the same config shape withalgorithm = "grpo" or "opd".
GRPO optimizes against your environment’s score_response reward; OPD has a
managed teacher grade your model’s own completions, using epochs over the
retained prompt pool with the step count derived for you, and warm-starts best
from a finished SFT run via init_from_adapter. See
Choose a training algorithm.
When Flash reads them: same as sft.toml. Keep the configs you need and
pick one at train time.
TRAINING.md
What it is: a playbook for the AI coding agent you point at this project, including the OPD teacher preflight, rollout reasoning budgets, reward design, run interpretation, and common Flash issue mitigations. When it travels: if you publish the whole scaffolded folder,flash env push includes .md sidecars, so
TRAINING.md can travel with the environment source in the Hub for humans and
coding agents.
The scaffolded environment.py is enough to publish on its own. Once your task
grows data files or helper modules, move it into a folder with environment.py
at the root and publish the whole folder. See
Structure the package.
Next steps
Single-turn environments
Fill in the environment class the scaffold starts you with.
Training
Point a config at your environment and submit a run.