> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.toloka.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Quality control

> How the platform checks labeled data — the quality criteria each output is judged against, who reviews the work, the verdicts you get back, what a failed check triggers, and what review costs.

Quality control is on by default on labeling nodes. You define criteria for acceptable labels and choose a review path. Quorum also offers a [no-review mode that relies on offboarding rules](#offboarding-rules-on-quorum-nodes).

![Quality control: the criteria the agent writes feed a reviewer — an LLM or a person, chosen by the review path — which turns a labeled item into per-criterion verdicts; a failing verdict sends the item back to be redone or regenerated.](https://mintcdn.com/toloka-platform/e2y6apyB8pyO00uJ/images/explanation/quality-control/quality-flow.svg?fit=max\&auto=format\&n=e2y6apyB8pyO00uJ\&q=85\&s=f9084aac5f020df37c9e41b60e996ad5)

The nodes differ in who reviews the work and what a failed check sets in motion.

## Quality criteria

A quality criterion is a named check on one output field. It defines a pass condition and a fail condition. It can also include an instruction of up to 5,000 characters seen only by the LLM reviewer.

The agent writes the criteria from the task you describe, then you review them and either tell the agent what to change or edit them by hand on the node's **Quality Requirements** card.

A node cannot deploy until its criteria clear a floor enforced by [Self-check](https://docs.platform.toloka.ai/docs/data/concepts/self-check.md): at least one criterion, each with a name filled in, each targeting one of the node's own output fields, and each pass and fail condition at least 10 characters long. On a Generation node, whether experts or an LLM produce the labels, every output field must be covered by at least one criterion. The Quorum node relaxes this requirement, so you can check a subset of its outputs.

## Who reviews the work

The review path is who checks the labels against the criteria. It is a per-node choice, and the paths a node offers depend on its kind:

| Node | Review paths available |
| - | - |
| Expert labeling (Generation) | LLM review, human review, or both |
| Quorum | LLM review, or none (offboarding rules instead) |
| Synthetic labeling | LLM review only |

LLM review is the default wherever it is available, and it grades every item. Human review, available on the expert-labeling node, checks a sample rather than every item: you set the share reviewed, from 10% to 100%, and the time a reviewer is given per item, at least 30 seconds or 3 seconds per quality criterion, whichever is greater. When both paths run together, the LLM grades every item and human reviewers double-check a sample. Synthetic labeling is locked to LLM review because it uses no experts. The Quorum node's no-review path is covered under [offboarding rules](#offboarding-rules-on-quorum-nodes).

## The review model

When an LLM reviews the work, it uses a review model. This per-node setting is separate from any model that generates synthetic labels. The platform has a short list of models, each with its own per-call price, and preselects a recommended one. The agent keeps that default when it builds the node. To use a different model, open the node and pick it under its **Quality Assurance** settings. On a Quorum node, you can also ask the agent to change the review model. On a Generation node, change it yourself under **Quality Assurance**. The choice covers all criteria on the node. The node's [exam](https://docs.platform.toloka.ai/docs/data/reference/nodes/generation.md#exam) uses the same model for the checks that need one; an exam answer compared directly against the golden answer needs no model. When the review mode names no model — human-only review, or a Quorum node with no reviewer — those exam checks use the recommended default.

## Verdicts

Every reviewed criterion returns a verdict with a written reason. The verdict is pass or fail unless the reviewer cannot reach either one. In that case, the verdict gives one of these reasons: the item could not be processed, the submitted data was not valid input, a field the check needed could not be read, the check itself failed, or the check was skipped.

The verdict's shape follows the output it covers:

* a plain output returns one verdict per criterion.
* an output that repeats within an item returns a list of verdicts, one per repetition.
* on a Quorum node, each criterion returns one verdict per expert, and a repeating output returns a list per expert.
* when LLM and human review both run on a node, each criterion returns one verdict from each, side by side.

Alongside the verdicts, a reviewed item carries a **review metadata field** with details about the review. This extra output field is named after the node and ends in `_qa_metadata`.

## When a check fails

A failing verdict can send the work back.

On expert-labeled nodes, a rejected submission is returned for correction, or reassigned, so the item gets another pass before it counts as done. The platform handles this rework automatically.

Under synthetic labeling, a label that fails its check is generated again and rechecked, up to one further attempt (two attempts total). Those retries consume billable tokens like any other generation, and total token spend stays within the run's contracted maximum. See [Label with an LLM instead of experts](https://docs.platform.toloka.ai/docs/data/guides/synthetic-labeling.md) for how this shows up in the results.

## Offboarding rules on Quorum nodes

To use Quorum without a reviewer, choose **No QA, Offboarding Only**, set an overlap of at least 3, turn off the per-expert task cap, and add at least one offboarding rule on the **Offboarding** card. Quality is then enforced by offboarding experts who persistently disagree with the majority on a chosen single-label output field. Answers are not graded, rejected, or changed.

See [Offboarding rules](https://docs.platform.toloka.ai/docs/data/reference/nodes/quorum.md#offboarding-rules) for the settings.

## What quality control costs

Review costs depend on the path and how much reviewing happens.

| Path | What is billed | Main multipliers |
| - | - | - |
| LLM review | One model call per check, at the review model's per-call price | items × criteria × repetitions × experts |
| Human review | Each sampled review, at a per-review price | sample rate × expected reviews per item |
| Quorum offboarding rules | No separate charge | — |

The human-review price uses the per-task price you set, multiplied by review time divided by labeling time, plus a 10% premium. An item sent back for rework is reviewed more than once, so the expected reviews per item can exceed one.

For what **Expected** and **Maximum** mean on these lines, see [Pipeline pricing](https://docs.platform.toloka.ai/docs/data/concepts/pricing.md). For the exact per-call and per-review figures, see the [Generation node](https://docs.platform.toloka.ai/docs/data/reference/nodes/generation.md) and [Quorum node](https://docs.platform.toloka.ai/docs/data/reference/nodes/quorum.md) references.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.