> ## Documentation Index
> Fetch the complete documentation index at: https://docs.freesolo.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom training client

> Operations, tensor alignment, limits, and recovery for custom training scripts.

Flash passes a managed client to your `main(client)` function. An async entrypoint
receives awaitable methods; a synchronous entrypoint uses blocking methods with
the same arguments. Start with [Custom algorithms](https://docs.freesolo.co/guides/custom-algorithms.md).
Install `freesolo-flash[custom]` for local PyTorch losses.

## State and sequencing

| Property | Meaning |
| - | - |
| `step` | Number of completed optimizer updates; restored on durable resume. |
| `restored_state` | Saved JSON algorithm state, such as a dataset cursor. |
| `config` | Authored run configuration as a dictionary; use `status()` for effective session limits. |
| `model_version` | Version used to order model mutations. |
| `session_id` | Identity of this GPU session. |

Operations are serialized within a session. Several backward calls can
accumulate gradients, but async calls do not create simultaneous optimizer
updates. Await an operation's result before using it. Cancellation of a local
await does not prove that the remote mutation was cancelled.

## Token data

```python theme={null}
from flash.custom import Datum

row = Datum(tokens=[10, 20, 30], weights=[0.0, 1.0])
```

Position zero predicts token 20 from token 10; position one predicts token 30
from the first two tokens. Each row has at least two token IDs and at most
`train.max_context_tokens`. Weights have `len(tokens) - 1` entries. Set weights
for prompt targets to zero. There is no automatic averaging: divide by the
number of tokens, examples, or other denominator your objective requires.

With `target_tokens`, supply a rectangular `[len(tokens)-1, K]` matrix where
`K` is 1–64. Weights and returned scores have that same shape. See
[soft targets](https://docs.freesolo.co/guides/custom-algorithms.md#train-with-soft-targets).

## Operations

| Call | Result or effect |
| - | - |
| `encode(text)` | Returns a list of token IDs; `decode(tokens)` returns text. |
| `sample(prompt, max_tokens=64, temperature=1.0, num_samples=1)` | Returns `sequences` with completion `tokens`, `text`, policy `logprobs`, `sampling_logprobs`, and `finish_reason`. Input plus output must fit the context limit. |
| `forward(data, top_k=0)` | Returns `outputs`, one per row, containing target `logprobs`. With positive `top_k`, also returns model-selected `top_logprobs` entries with `token` and `logprob`. |
| `forward_backward(data, metrics=None)` | Accumulates gradients of `-sum(weights * logprobs)`; returns per-row scores and metrics including `surrogate_loss`. Explicit weights are required. |
| `forward_backward_custom(data, loss_fn)` | Calls a local differentiable loss on scores, then accumulates the corresponding remote gradients. |
| `optim_step(learning_rate=..., max_grad_norm=1.0, algorithm_state=None)` | Clips/applies accumulated gradients, clears them, and advances the step. Requires a prior backward operation. |
| `zero_grad()` | Discards accumulated gradients without advancing the optimizer. |
| `save_checkpoint(name, algorithm_state=None)` | Saves a completed step after gradients have been applied or cleared. Names are immutable; automatic `step-N` names are reserved. |
| `status()` | Returns current session, model, step, and input-limit information. |
| `logs(after=0)` | Reads operation logs after a cursor. |
| `retry_pending()` | Reconciles an operation with an ambiguous response before another operation is sent. |

`load_checkpoint` is not supported for managed sessions. Start a new run with
`custom.resume_from` to restore a durable step, or `train.init_from_adapter`
for a fresh optimizer initialized from an adapter.

A custom loss has the signature `loss_fn(data, logprobs)` and returns
`(scalar_tensor, metrics)`. Each tensor has shape `[T]`, or `[T, K]` for explicit
targets, and supports local autograd. Return a finite scalar and finite numeric
metrics. Cross-sequence losses can compare multiple rows; do not detach the
scores or convert the loss to a Python float before returning it.

## Request limits

| Input | Limit |
| - | - |
| Rows per batch / samples per call | 16 |
| Explicit target candidates per position | 1–64 |
| Explicit target entries per request | 100,000 |
| `forward(top_k=K)` | K from 0 to 20; at most 100,000 returned alternatives across the batch |
| Encoded text | 100,000 characters per call |
| Request body | 4 MiB |
| Metrics | At most 64 finite numeric values; names 1–128 characters |
| `learning_rate` | 1e-12 through 1 |
| `max_grad_norm` | 1e-12 through 1,000 |
| Sampling temperature | 0 through 10 |
| Algorithm state | JSON-serializable object, at most 1 MiB |
| Checkpoint name | 1–64 letters, digits, underscores, or hyphens; first character alphanumeric |

These protocol limits are separate from memory admission. Full-vocabulary
logits can exhaust GPU memory even with a small number of target candidates.
Reduce sequence length or row size when an operation exceeds available memory.
The [configuration reference](https://docs.freesolo.co/reference/configuration.md#custom) specifies
run-level limits and the release dependency for larger custom shapes.

## Failure and resume

If an operation's outcome is unresolved, use `retry_pending()` rather than
reissuing it as a new update. Do not replay a mutation into a different session.
A failed session requires a new run from a durable checkpoint.

Pass your cursor/state to `optim_step(algorithm_state=...)` so automatic saves
include it, then resume from `client.step` and `client.restored_state`. Finishing
the Python function does not add optimizer steps automatically. Keep the
controller running until your intended horizon is reached; Ctrl-C cancels a
custom run. See [durable resume](https://docs.freesolo.co/guides/custom-algorithms.md#resume-a-checkpoint).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.