Evaluation tasks
An evaluation task combines a dataset, evaluation target, evaluators, and field mappings. Task versions save the configuration. Each execution creates a Run containing its status, frozen snapshot, and results.
For users who start, track, and analyze evaluations. Verification baseline: the project's current implementation as of 2026-10-30. Entry points in deployed environments may vary by version and permissions.
Search keywords: Evaluation Run, draft, task version, run snapshot, copy, cancel.
1. Choose by goal
- Configure an evaluation: Create evaluation tasks and map fields.
- Interpret scores and locate failures: View results and execution traces.
- Repeat evaluations on a fixed schedule: Scheduled tasks.
2. Find and continue tasks
- Go to Evaluation tasks and find a task by searching its name or filtering by status.
- For tasks with incomplete configurations, use Continue configuration, complete the settings, and then start. Saving a draft does not call the evaluation target or scoring model.
- For a task that has run, click its name to open Task results, then switch between Results and Settings. Running tasks refresh their status automatically.
3. Understand statuses
| Status | Meaning and action |
|---|---|
| Not started / Draft | The configuration has not been executed; continue configuring or save a draft. |
| Queued | Submitted and waiting for execution resources. Check progress first to avoid duplicate creation. |
| Running | Calling the evaluation target or scoring. |
| Cancelling | A cancellation request has been submitted; waiting for the current execution to settle. |
| Completed | All Cases have finished with no failed Cases. This does not imply that business quality requirements are met. |
| Partial | Some calls or scoring attempts failed; completed results remain viewable. |
| Failed | The execution failed. Check Cases and failure reasons to locate the issue. |
| Cancelled | Further execution has been canceled; completed results are retained. |
4. Copy, cancel, and delete
Using Copy Task in the list immediately creates an independent draft with a default copy name and the configuration retained for review. It does not start automatically. Check the dataset version, evaluation target, evaluator versions, and mappings before and after copying. After the success confirmation, open the configuration to complete it and start.
To stop a running task, click Cancel on the results page and confirm. Cancellation does not erase completed results or reverse external actions the Agent under test has already performed.
Only draft or failed tasks can be deleted from the list. Confirm that you have saved any required results before deleting; the operation cannot be undone. Independent tasks within the same project must have unique names, including drafts. If a name conflicts, change it and continue. You do not need to reenter the entire configuration.
5. Versions and historical facts
A task binds to specific dataset and evaluator versions, then freezes an execution snapshot at startup. Later asset edits do not rewrite historical inputs, mappings, or scores. When analyzing historical tasks, use the frozen content in Settings. Do not substitute an asset's current content for the configuration actually used in that run.
6. Next steps and related pages
- Create a task: Create evaluation tasks and map fields.
- Find low-scoring Cases: View results and execution traces.
- Repeat evaluations: Scheduled tasks.
Return to the manual home page:

