> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.toloka.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Prepare training data from labels

> Convert a Data Labeling export into chat training rows while keeping a deterministic evaluation set out of Training.

This guide converts the result of one Data Labeling pipeline run into the chat JSONL that Training reads. The worked mapping is for a classification pipeline with one text input named `ticket_text` and one string output named `intent`.

The conversion creates three files:

* `training.jsonl` contains the rows you upload for Training.
* `evaluation-inputs.jsonl` contains prompts to send to the trained model later.
* `evaluation-gold.jsonl` contains the expected labels and stays local.

> [!WARNING]
>
> Upload only `training.jsonl`. Uploading either evaluation file, or copying its labels into the training rows, leaks the answers you intend to use for final evaluation.

## Before you start

You need:

* A completed pipeline run whose End node returns the label you want the model to learn.
* The run's JSON export. Follow [Export your labeled results](https://docs.platform.toloka.ai/docs/data/guides/export-results.md) and wait for the run to finish before starting the export.
* Python 3.9 or later for the local conversion.
* The exact input and output field names in the export. See [Export format](https://docs.platform.toloka.ai/docs/data/reference/export-format.md) if you need to inspect the entry shape.

Use one run export at a time. The `index` is unique within a run, but it is not a global item identifier across several runs.

## Map the export to messages

For this classification example, each completed export entry maps as follows:

| Export value | Training or evaluation value | Purpose |
| - | - | - |
| `input.ticket_text` | A `user` message's `content` | The text the model sees |
| `output.intent` | The final `assistant` message's `content` | The answer the model learns |
| `index` | The evaluation join key | Matches a prediction to its held-out answer |
| `status` | A filter | Only `completed` entries are eligible |
| `metrics` | Not copied | Timing, attempts, and annotator identifiers are not model inputs |

A completed entry can look like this:

```json theme={null}
{
  "index": 41,
  "status": "completed",
  "input": { "ticket_text": "I was charged twice for one subscription." },
  "output": { "intent": "complaint" }
}
```

It becomes this training row when it is assigned to the training partition:

```json theme={null}
{"messages":[{"role":"user","content":"I was charged twice for one subscription."},{"role":"assistant","content":"complaint"}]}
```

Do not convert `failed`, `cancelled`, `paused`, `in_progress`, or `idle` entries. They do not represent a completed input-to-label pair. The script below also stops if a completed entry lacks the selected string input or label, rather than silently creating a partial example.

> [!NOTE]
>
> A Quorum output can contain per-expert arrays. Point `--label-field` at the resolved scalar output your End node returns for training. Do not stringify an answer array or use `annotatorIds` as the label unless that is deliberately the task you are training.

## Save the converter

Save the following as `prepare_label_export.py`:

```python theme={null}
import argparse
import hashlib
import json
from pathlib import Path


def write_jsonl(path, rows):
    path.write_text(
        "".join(json.dumps(row, ensure_ascii=False) + "\n" for row in rows),
        encoding="utf-8",
    )


parser = argparse.ArgumentParser()
parser.add_argument("export", type=Path)
parser.add_argument("--input-field", required=True)
parser.add_argument("--label-field", required=True)
parser.add_argument("--evaluation-count", type=int, default=2)
parser.add_argument("--output-dir", type=Path, default=Path("prepared"))
args = parser.parse_args()

rows = json.loads(args.export.read_text(encoding="utf-8"))
if not isinstance(rows, list):
    raise SystemExit("The export must be a JSON array.")
if args.evaluation_count < 1:
    raise SystemExit("--evaluation-count must be at least 1.")

eligible = []
seen_indexes = set()
skipped_statuses = 0

for row in rows:
    if not isinstance(row, dict):
        raise SystemExit("Every export entry must be a JSON object.")
    if row.get("status") != "completed":
        skipped_statuses += 1
        continue

    index = row.get("index")
    input_values = row.get("input")
    output_values = row.get("output")
    if not isinstance(index, int):
        raise SystemExit("Every completed entry must have an integer index.")
    if index in seen_indexes:
        raise SystemExit(f"Duplicate completed index: {index}")
    seen_indexes.add(index)

    if not isinstance(input_values, dict) or not isinstance(output_values, dict):
        raise SystemExit(f"Completed entry {index} has no input or output object.")
    prompt = input_values.get(args.input_field)
    label = output_values.get(args.label_field)
    if not isinstance(prompt, str) or not prompt.strip():
        raise SystemExit(
            f"Completed entry {index} has no non-empty string input field "
            f"{args.input_field!r}."
        )
    if not isinstance(label, str) or not label.strip():
        raise SystemExit(
            f"Completed entry {index} has no non-empty string label field "
            f"{args.label_field!r}."
        )
    eligible.append((index, prompt.strip(), label.strip()))

if args.evaluation_count >= len(eligible):
    raise SystemExit("The evaluation count must leave at least one training row.")

groups = {}
for record in eligible:
    groups.setdefault(record[1], []).append(record)

ranked_groups = sorted(
    groups.values(),
    key=lambda group: (
        hashlib.sha256(group[0][1].encode("utf-8")).digest(),
        min(record[0] for record in group),
    ),
)
evaluation = []
for group in ranked_groups:
    if len(evaluation) >= args.evaluation_count:
        break
    evaluation.extend(group)

evaluation_indexes = {record[0] for record in evaluation}
training = [record for record in eligible if record[0] not in evaluation_indexes]
if len(training) < 10:
    raise SystemExit(
        "This split leaves fewer than 10 rows to upload for Training. Add completed labels or "
        "reduce --evaluation-count."
    )

training.sort(key=lambda record: record[0])
evaluation.sort(key=lambda record: record[0])
args.output_dir.mkdir(parents=True, exist_ok=True)

write_jsonl(
    args.output_dir / "training.jsonl",
    (
        {
            "messages": [
                {"role": "user", "content": prompt},
                {"role": "assistant", "content": label},
            ]
        }
        for _, prompt, label in training
    ),
)
write_jsonl(
    args.output_dir / "evaluation-inputs.jsonl",
    (
        {
            "index": index,
            "messages": [{"role": "user", "content": prompt}],
        }
        for index, prompt, _ in evaluation
    ),
)
write_jsonl(
    args.output_dir / "evaluation-gold.jsonl",
    (
        {"index": index, "expected": label}
        for index, _, label in evaluation
    ),
)

print(f"Eligible completed rows: {len(eligible)}")
print(f"Rows to upload for Training: {len(training)}")
print(f"Evaluation rows: {len(evaluation)}")
print(f"Skipped non-completed rows: {skipped_statuses}")
```

The split is reproducible. It ranks prompts by a SHA-256 digest and selects whole prompt groups for evaluation until it reaches the requested count. Exact duplicate prompts therefore cannot land in both partitions. Because duplicate prompts stay together, the actual evaluation count can be greater than `--evaluation-count`.

The script requires at least 10 rows in `training.jsonl` after the local evaluation split. When you create a Training from the new dataset, the default 80/10/10 split assigns those 10 rows as 8 train, 1 validation, and 1 test. The 8 train rows fill the default batch size. Custom splits or larger batch sizes can require more rows.

## Run the conversion

Run the converter with the field names from your export:

```bash theme={null}
python3 prepare_label_export.py labeled-results.json \
  --input-field ticket_text \
  --label-field intent \
  --evaluation-count 2 \
  --output-dir prepared
```

A run with 14 eligible entries, unique prompts, and 2 unfinished entries prints:

```text theme={null}
Eligible completed rows: 14
Rows to upload for Training: 12
Evaluation rows: 2
Skipped non-completed rows: 2
```

Review the counts. An error naming a completed entry usually means a field name is wrong or the selected output is not a scalar string.

## Inspect the separation

A line in `evaluation-inputs.jsonl` contains the prompt but not its answer:

```json theme={null}
{"index":41,"messages":[{"role":"user","content":"I was charged twice for one subscription."}]}
```

The matching line in `evaluation-gold.jsonl` keeps the answer separate:

```json theme={null}
{"index":41,"expected":"complaint"}
```

Keep `evaluation-gold.jsonl` away from the inference request. The `index` is only a local join key for scoring predictions later.

## Upload the training partition

For this step, install the [CLI](https://docs.platform.toloka.ai/docs/cli.md), sign in, and select a project where your credentials can create and upload datasets. For browser login, select a team with **Full access** using the [team switcher](https://docs.platform.toloka.ai/docs/core/teams-and-balance.md#switch-teams). For an API key, check [CLI permissions](https://docs.platform.toloka.ai/docs/cli.md#permissions).

Create a dataset in the project selected by the CLI, upload only the training file, and copy the dataset ID printed by `dataset list`:

```bash theme={null}
toloka dataset create support-ticket-training
toloka dataset upload support-ticket-training ./prepared/training.jsonl
toloka dataset list
```

Dataset uploads append rows. Do not repeat the upload to the same dataset unless you intend to add a second copy of every training example.

The converter produces a `messages` array with one user turn and a final assistant label. For other conversation formats, see [Training data requirements](https://docs.platform.toloka.ai/docs/train/concepts/how-training-works.md#data).

A cost estimate validates the selected rows before a Training Run can start.

## Keep the two evaluation boundaries separate

The local evaluation set stays separate from the dataset slice used by Training:

* Keep `evaluation-inputs.jsonl` and `evaluation-gold.jsonl` out of Training.
* Create the Training with `--dataset` pointing to the dataset containing only `training.jsonl`. The command creates an 80% train, 10% validation, 10% test split for this new dataset and freezes a slice.
* The platform learns from the train part and reports validation metrics on the validation part. The test part stays out of learning.
* Keep the local evaluation set untouched until you have actual predictions from the downloaded Model Version.

The same chat rows can be used for [fine-tuning](https://docs.platform.toloka.ai/docs/train/guides/fine-tune.md) or [gisting](https://docs.platform.toloka.ai/docs/train/guides/gist.md). Gisting additionally requires a prompt text file. The complete label-to-training-to-evaluation path is in [Label, train, and evaluate](https://docs.platform.toloka.ai/docs/workflows/label-train-evaluate.md).

## Adapt the mapping

For a pipeline with several inputs, construct one user message that names each field consistently. For a generative output, put the exact target text in the final assistant message and choose an evaluation metric suited to that output. Do not copy export metadata into the conversation unless it is information the model should receive at inference time.

If separate source items can contain semantically equivalent text with different wording, deduplicate or group them using a stable source identifier before conversion. The example only guarantees that identical prompt strings stay in one partition.

## Related

* [Export format](https://docs.platform.toloka.ai/docs/data/reference/export-format.md)
* [Export your labeled results](https://docs.platform.toloka.ai/docs/data/guides/export-results.md)
* [How Training reads data](https://docs.platform.toloka.ai/docs/train/concepts/how-training-works.md#data)
* [Fine-tune a model](https://docs.platform.toloka.ai/docs/train/guides/fine-tune.md)
* [Compress a prompt with gisting](https://docs.platform.toloka.ai/docs/train/guides/gist.md)
* [Label, train, and evaluate](https://docs.platform.toloka.ai/docs/workflows/label-train-evaluate.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.