> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.toloka.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Create and manage datasets

> Create and manage project datasets shared by Data Labeling and Train, including upload rules, fields, filters, exports, and Training conversation rows.

A [dataset](https://docs.platform.toloka.ai/docs/start/glossary.md#dataset) stores project data for both products. A Data Labeling pipeline reads items and writes labeled fields; a Training draft reads conversation rows from a frozen dataset slice for fine-tuning or gisting. You manage a dataset from its page in the project sidebar or by asking the agent in chat. For the field, item, and value model, see [Fields and data](https://docs.platform.toloka.ai/docs/data/concepts/fields-and-data.md).

## Before you start

You need:

* A [project](https://docs.platform.toloka.ai/docs/core/projects.md). A dataset lives inside one project. Pipelines and Trainings in that project can select it.
* Your items in a JSON, JSONL, or CSV file, if you want to add data now. The next section gives the exact shape.

## Create a dataset

Two paths create a dataset:

* **On the dataset page.** In the project sidebar, open the **Datasets** section and click the plus button (tooltip **Create dataset**). The dataset opens with a placeholder name (**New Dataset**, numbered if one already exists), and its page is the upload zone. The first file you upload renames the dataset to that file's name.
* **Through the agent.** Attach a JSON, JSONL, or CSV file in chat and ask the agent to build a dataset from it. The agent creates the dataset and loads the file in one step.

An empty dataset page is titled **Add File**. Drag a file onto it, or click **Select file...** to choose one. The zone accepts JSON, JSONL, and CSV, chosen by the file extension; anything else is refused with `Please upload a JSON, JSONL or CSV file.`

> [!NOTE]
>
> A dataset the agent builds keeps the name the agent gives it. Only the dataset page's first upload renames the dataset to the file's name. A large file can take a while to load. If the agent's upload runs long, it keeps loading in the background while the agent moves on.

## What the file must look like

A JSON file is an array of objects, with one object per item:

```json theme={null}
[
  { "ticket_text": "Hi, how do I reset my password? I can't find the option anywhere." },
  { "ticket_text": "This is the third time my order arrived damaged. I'm furious." },
  { "ticket_text": "Just wanted to say your support team was incredibly helpful, thank you!" }
]
```

A JSONL file contains one JSON object per nonempty line, without an enclosing array. Blank lines are skipped. For example, save these two lines as `tickets.jsonl`:

```jsonl theme={null}
{"ticket_text":"How do I reset my password?"}
{"ticket_text":"My order arrived damaged."}
```

The upload screen states three limits:

* Up to 1024 MB per file.
* Up to 1024 KB per item.
* Up to 150,000 items in a dataset. The cap counts everything the dataset holds, across the first upload and every append — the screen says "items in file", but the limit is per dataset.

A CSV file starts with a header row, and each later row becomes one item keyed by those headers. Headers must be present, non-empty, and unique, and every row must have as many values as the header has columns.

> [!WARNING]
>
> Every value from a CSV arrives as text — `"age": "30"`, never `30`. Nothing coerces types later in the pipeline, so a field declared as a number or a boolean over CSV data silently skips every row. Declare CSV-derived fields as text.

Each item is stored whole in a built-in field named `uploaded`, which holds its raw value. The fields you actually filter and label on come from that raw value in the next section.

An upload is all-or-nothing: a file with any invalid item adds nothing to the dataset, and the upload lists up to 50 of the problems for you to fix. Unsupported formats, empty files, files over 1024 MB, and items over 1 MB are rejected. A `.json` file must have an array root; each nonempty line in a `.jsonl` file must be a JSON object. The one exception: a valid file that overflows the 150,000-item cap is imported up to the cap, and the upload reports `Dataset item cap reached; remaining rows were not imported.` The exact messages, with their fixes, are on the [errors reference](https://docs.platform.toloka.ai/docs/data/reference/errors.md#on-the-dataset-page).

## Use a dataset for Training

Training reads one chat example from each selected row. For nested conversations, use JSON or JSONL so `messages` remains an array rather than the text value a CSV upload produces:

```json theme={null}
{
  "messages": [
    { "role": "user", "content": "Classify this ticket: I cannot reset my password." },
    { "role": "assistant", "content": "account_access" }
  ]
}
```

Each row needs a `messages` array with at least one user turn and a final assistant turn containing the expected output. For system messages, tool calls, and other sequence rules, see [Training data requirements](https://docs.platform.toloka.ai/docs/train/concepts/how-training-works.md#data).

A dataset can instead keep the conversation in a `messages` field and the expected answer in a label field; the platform appends that label as the final assistant turn for Training. Follow [Prepare training data from labels](https://docs.platform.toloka.ai/docs/train/guides/prepare-training-data.md) to convert a labeling export into valid Training rows.

Create a Training with `--dataset <name-or-id>` to prepare its split column and freeze a slice. A dataset without a split column gets an 80% train, 10% validation, 10% test split. Training learns from parts with the `train` role, uses parts with the `validate` role for validation, and keeps test rows out of learning. A fresh 10-row dataset gives 8 training rows, 1 validation row, and 1 test row.

The slice fixes the selected rows and their field values. After appending rows, run `toloka training update <training> --dataset <dataset>` to place them in the split and freeze a new slice, then calculate a new estimate. Items cannot be edited or removed, so to replace bad rows, upload corrected rows to a new dataset and pass that dataset instead. To reuse a ready slice or choose split parts, see [Training dataset selections](https://docs.platform.toloka.ai/docs/train/reference/settings.md#dataset-selections).

## Turn the raw upload into fields

The platform reads the items during upload and automatically proposes fields for them. When the first upload finishes, the **Infer Fields** modal opens on its own, titled *Create fields from upload file*.

![The Create fields from upload file modal: the inferred fields listed with a checkbox each, the Select all control, and the Create button.](https://mintcdn.com/toloka-platform/e2y6apyB8pyO00uJ/images/how-to/manage-datasets/create-fields-modal.png?fit=max\&auto=format\&n=e2y6apyB8pyO00uJ\&q=85\&s=76f894e5e5ece29d8dabde4e199a1031)

The modal lists the proposed fields with a checkbox each and a **Select all** control. Tick the ones you want and click **Create**. Each one becomes a field that reads its value from `uploaded`. The columns fill in afterward. Values populate in the background, so a large dataset's fields appear gradually. Names already taken in the dataset show struck through in the modal. You can reopen the modal any time with the **Infer Fields** button. A dataset holds at most 500 fields, and one field holds at most 500 recorded values per item.

The platform skips an item when its raw value has nothing at the field's location or holds a value that does not fit the field's type. The item does not fail: it keeps its row and its `uploaded` value, and has no value for that field. Every later upload re-runs these derivations on the new items automatically, so appended items pick up the same fields.

> [!NOTE]
>
> The modal derives new fields that read from `uploaded` at a fixed location. That is the dataset page's only field-deriving path. The agent can go further: it can read from any [location](https://docs.platform.toloka.ai/docs/data/concepts/fields-and-data.md) inside the value, give a field its own type, read from a field other than `uploaded`, and write into a field that already exists. A field name the agent sets must be 1 to 128 characters, start with a lowercase letter, use only lowercase letters, numbers, and underscores, and end with a letter or number.

## Append more items

Once a dataset has one validated upload, an **Append items** button appears next to **Infer Fields**. It opens the same upload zone, titled *Append items to dataset*. An appended file goes through the same validation as the first. One upload runs at a time per dataset, and a truncated file that stops mid-item fails the upload.

Appended items continue the same numbering: item indexes start at 1 and never reset, so a new upload's items take the next numbers in sequence.

> [!WARNING]
>
> An appended file must stay schema-compatible with the first upload's fields. If a field is missing, or its type or values no longer fit, the whole append fails and no items are added. See the schema-compatibility messages on the [errors reference](https://docs.platform.toloka.ai/docs/data/reference/errors.md#on-the-dataset-page). Appends are never deduplicated, so uploading the same items again adds them again.

The agent appends to an existing dataset the same way, from a file you attach in chat.

## Browse and filter

The dataset page lists its items under an **Items (N)** header showing the total count, 100 items per page, with **Prev** and **Next**. The table has one column per field, plus an index column. Click a column's field name to see its JSON schema, its description, its field ID, and its largest stored revision.

A cell shows a preview truncated at 120 characters. A value too large to preview shows a *view value* link and loads when you click it. A field with more than one recorded value is marked `(N rev)`; opening it lists every value, latest first.

To filter items:

1. Open the funnel on the field's column.
2. Set **JSONPath** to a [location](https://docs.platform.toloka.ai/docs/data/concepts/fields-and-data.md) in the value, choose `=` or `≠` under **Operator**, and enter a **Value**.
3. Optionally select **Latest revision only** to match each item's newest recorded value, choose a [source](https://docs.platform.toloka.ai/docs/start/glossary.md#source) under **Created by**, or select **No revision for this field** to find items with no value for it.
4. Click **Apply**.

`≠` with an empty **Value** matches any existing value. **Clear** removes the filter. The Start node uses the same filters to choose which items a pipeline run launches; see the [Start node's item filter](https://docs.platform.toloka.ai/docs/data/reference/nodes/start-and-end.md#item-filter).

Value filtering is unavailable for a field whose largest value is over 32 KB. A notice says so, and you can still filter that field by source or by missing value.

## Export fields to file

The **Export fields to file** section on the dataset page exports the dataset's own fields. Pick the fields to include and click **Export selected fields**. The platform generates a JSON file in the background. When its badge turns to `completed`, use the **Download** link on that export's row.

The agent can export too, writing the fields you name to a file in its workspace. To export a pipeline run's results, use the run's Items tab instead. See [Export your labeled results](https://docs.platform.toloka.ai/docs/data/guides/export-results.md).

## Rename or hide a dataset

The ⋯ menu on the dataset header has **Rename dataset** and **Hide Dataset** — on an already-hidden dataset the second item reads **Show Dataset**, and a `hidden` badge sits next to the name. Renaming opens an *Edit dataset name* dialog. The agent can rename a dataset too.

Hiding a dataset removes it from the sidebar's **Datasets** list. The dataset still exists and still feeds any pipeline bound to it, but it becomes invisible to the agent: every agent operation on a hidden dataset fails until you unhide it. To see hidden datasets again, turn on **Show hidden datasets** (the eye toggle on the **Datasets** section). Hiding is a dataset-page action. The agent cannot hide a dataset.

## What this does not do

* **No hand-editing of item values.** The table is read-only, and nothing changes a value an item already holds. A new value arrives from an upload, a field derivation, a pipeline run writing its results back, or the agent importing values from a file into fields that are still empty — already-populated pairs are skipped.
* **No deleting items or datasets.** You can hide a dataset, but nothing removes a dataset or an individual item.
* **Pipelines and Trainings select rather than copy datasets.** A [Start node](https://docs.platform.toloka.ai/docs/data/reference/nodes/start-and-end.md) binds a pipeline to a project dataset and chooses the items a run launches. A Training draft reads train and validation parts from one frozen dataset slice.

## Related

* [Prepare training data from labels](https://docs.platform.toloka.ai/docs/train/guides/prepare-training-data.md) — turn labeled results into valid Training rows
* [Fields and data](https://docs.platform.toloka.ai/docs/data/concepts/fields-and-data.md) — understand the field, item, and value model
* [Start and End nodes](https://docs.platform.toloka.ai/docs/data/reference/nodes/start-and-end.md) — bind a pipeline to a dataset and filter launch items
* [Fine-tune a model](https://docs.platform.toloka.ai/docs/train/guides/fine-tune.md) — use labeled conversations for supervised fine-tuning
* [Compress a prompt with gisting](https://docs.platform.toloka.ai/docs/train/guides/gist.md) — use examples and a prompt for gisting
* [Label a large dataset in batches](https://docs.platform.toloka.ai/docs/data/guides/label-at-volume.md) — upload each batch as its own dataset
* [Get your first labeled dataset](https://docs.platform.toloka.ai/docs/data/get-started/first-labeled-dataset.md) — follow the attach-and-build flow end to end
* [Dataset and pipeline errors](https://docs.platform.toloka.ai/docs/data/reference/errors.md#on-the-dataset-page) — fix upload, append, and filter errors


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.