Local testing

@toloka-platform/sdk/testing exports runCodeNode, typed Vitest test helpers, and expect. Use them to test authored nodes and connected pipeline flows before applying changes. Import this entry point from Vitest test files. It sets up Vitest fixtures, so importing it from a plain Bun script fails.

Install test tools

Start with Bun and the SDK, then install project test dependencies:

bun add -d typescript @types/node vitest playwright
bunx playwright install chromium

Code-node tests require python3. Task UI tests require network access to the hosted labeling-interface launcher as well as the local Chromium download. They use no platform credentials and create no platform project. Use synthetic or redacted test data because the launcher executes with access to the data you supply. For a new project, save this as tsconfig.json:

tsconfig.json

{
  "compilerOptions": {
    "target": "ES2024",
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "strict": true,
    "skipLibCheck": true,
    "noEmit": true,
    "types": ["node"]
  },
  "include": ["pipelines/**/*.ts"]
}

Test a Code node

Create a node that trims whitespace with code(), then assemble the pipeline with definePipeline():

pipelines/normalize/pipeline.ts

import { code, connect, definePipeline, field, root } from "@toloka-platform/sdk";

const text = field("text", { type: "string" });
const normalized = field("normalized", { type: "string" });
const boundary = root({ started: [text], completed: [normalized] });

export const normalize = code("normalize", {
  about: { name: "normalize" },
  input: [{ field: text, optional: false }],
  output: [{ field: normalized, optional: false }],
  code: {
    code: 'async def main(inputs):\n    return {"normalized": inputs["text"].strip()}\n',
  },
});

export default definePipeline({
  dataset: "normalization-items",
  fields: [text, normalized],
  nodes: [boundary, normalize],
  connections: [
    connect(boundary.started, normalize.input),
    connect(normalize.output, boundary.completed),
  ],
});

Bind a test to that node with test.code. Use test.pipeline to test a flow, writing each step yourself: it does not execute graph connections. Call runCode for each Code node, or renderUi and submit for a labeling interface. Supply the input state, merge the declared outputs into that state, choose the branch to follow, and assert the completed fields. Use a representative local row, redacted when needed, and add separate scenarios for branches and failures. This pipeline has one Code node, so its flow test makes one runCode call:

pipelines/normalize/pipeline.spec.ts

import { expect, test } from "@toloka-platform/sdk/testing";
import pipeline, { normalize } from "./pipeline";

test.code(pipeline, normalize, "trims text", async ({ run }) => {
  await expect(run({ text: "  hello  " })).resolves.toEqual({ normalized: "hello" });
});

test.pipeline(pipeline, "produces the completed field", async ({ runCode }) => {
  const item = { text: "  hello  " };
  const output = await runCode(normalize, item);
  expect({ ...item, ...output }).toMatchObject({ normalized: "hello" });
});

run(inputs, options?) calls runCodeNode(node, inputs, options?). The runner executes the Python node’s async main(inputs) and returns its output object as Promise<Record<string, unknown>>.

Runner option	Default	Use
`env`	`{}`	Test-only environment values, including substitutes for secrets
`pythonExecutable`	`"python3"`	Python executable to start
`timeoutMs`	`10_000`	Maximum execution time

The runner inherits the process environment, adds authored static environment values, then applies env overrides. It does not fetch project secrets. Imports must be available to that Python executable; local package availability does not establish platform support. See the Code node runtime. Call runCodeNode directly for ordinary Vitest tests, including failure assertions. These tests supplement the typed node test:

pipelines/normalize/errors.spec.ts

import { CodeNodeExecutionError, expect, runCodeNode, test } from "@toloka-platform/sdk/testing";
import { normalize } from "./pipeline";

test("reports a missing input", async () => {
  await expect(runCodeNode(normalize, {})).rejects.toBeInstanceOf(CodeNodeExecutionError);
});

Test a labeling interface

Task UI is the testing API for a node’s labeling interface. Save this minimal generation() fixture as pipelines/label/pipeline.ts. It exercises the interface locally; configure audience, pricing, quality, and onboarding using the Generation node reference before applying it for labeling work.

pipelines/label/pipeline.ts

import { connect, definePipeline, field, generation, root } from "@toloka-platform/sdk";

const label = field("label", { type: "string" });
const boundary = root({ started: [], completed: [label] });

export const annotate = generation("annotate", {
  publicName: "Choose a label",
  entities: {
    input: [],
    output: [{ field: label, entityType: "SHORT_ANSWER" }],
  },
  ui: {
    code: '<Field.RadioGroup path="label" options={[{ label: "Accept", value: "accepted" }, { label: "Reject", value: "rejected" }]} />',
    data: "{}",
    preprocess: "return {};",
    postprocess: "return {};",
    constants: "{}",
  },
});

export default definePipeline({
  dataset: "label-items",
  fields: [label],
  nodes: [boundary, annotate],
  connections: [
    connect(boundary.started, annotate.input),
    connect(annotate.output, boundary.completed),
  ],
});

Use a .ui.spec.ts file beside the pipeline. A test.ui callback receives a render helper bound to its node:

pipelines/label/pipeline.ui.spec.ts

import { expect, test } from "@toloka-platform/sdk/testing";
import pipeline, { annotate } from "./pipeline";

test.ui(pipeline, annotate, "submits a label", async ({ render }) => {
  const task = await render();
  await task.setValue("label", "accepted");
  expect(await task.validationErrors()).toEqual([]);
  expect(await task.submit()).toMatchObject({ label: "accepted" });
});

test.pipeline(pipeline, "produces the completed label", async ({ renderUi }) => {
  const task = await renderUi(annotate);
  await task.setValue("label", "rejected");
  expect(await task.submit()).toMatchObject({ label: "rejected" });
});

Pass render({ data }) to supply input values. Each session gets a fresh browser context and page, with automatic cleanup after the test. One Chromium process is shared per Vitest worker.

Session method	Purpose
`componentTree()`	Inspect the rendered component hierarchy.
`componentDocumentation(names?)`	Read supported component props and value shapes.
`setValue(path, value)` / `getValue(path)`	Address a field by its stable path.
`validationErrors()`	Read component validation errors.
`submit()`	Return submission data and validate the declared output entities.
`screenshot(path)`	Save a full-page PNG.
`close()`	Release a session early; repeated calls are safe.

Submission data can include echoed inputs; assert the output fields you need. preprocess and postprocess contain JavaScript function bodies, not TypeScript. Only data is in scope there; CONST is available in ui.code only. Use componentDocumentation when a component’s value is structured or array-valued, then test that shape through setValue, getValue, and submit. For an ordinary Vitest test, the ui fixture provides ui.render(pipeline, node, options?). ui is a callback fixture, not an export. Plain tests can inspect components or assert invalid submissions, but they do not satisfy required typed node coverage.

Errors and hardware coverage

Error	Meaning and useful properties
`CodeNodeExecutionError`	Python failed, timed out, or returned invalid/non-object JSON. Read `exitCode` and `stderr`.
`CodeNodeInfrastructureError`	The selected Python executable could not start.
`TaskUiInfrastructureError`	Browser or launcher setup failed.
`TaskUiValidationError`	Submission failed form validation, output-schema validation, or hardware coverage. Read `formErrors`, `entityErrors`, and `coverage`.

Hardware-backed components such as file pickers and cameras make task.coverage.complete false. By default, submit() rejects. To test only the available paths, pass allowPartialCoverage: true to render, inject values matching the component contract, and assert the resulting mapping. Hardware-backed output paths are excluded from entity validation. This does not verify capture, upload, or recording constraints; exercise them on a real device.

Run the local gate

Run the CLI from the project root:

toloka check

It loads pipeline source, runs project TypeScript, checks every labeling interface against bundled declarations, runs Vitest, and requires passing typed node and pipeline-flow tests. A typed Code test must execute its runner, assert a result, and pass. A typed UI test must submit successfully, assert a result, and pass.

Test wrapper	Default timeout
`test.code`	30 seconds
`test.ui`	60 seconds
`test.pipeline`	120 seconds

Pass { timeout: milliseconds } after the callback to change the test timeout. The code runner’s timeoutMs remains a separate limit. missing, failed, and infrastructure_error results block the gate. Hardware partial coverage is reported without blocking. toloka apply reruns the gate before resolving credentials or writing platform state. Platform node checks remain separate from these local tests.

TypeScript SDK

Pipelines as code

CLI

Install test tools

Test a Code node

Test a labeling interface

Errors and hardware coverage

Run the local gate

​Install test tools

​Test a Code node

​Test a labeling interface

​Errors and hardware coverage

​Run the local gate

Install test tools

Test a Code node

Test a labeling interface

Errors and hardware coverage

Run the local gate