Skip to main content

Evaluation tasks

Last updated 10/30/2026

An evaluation task combines a dataset, evaluation target, evaluators, and field mappings. Task versions save the configuration. Each execution creates a Run containing its status, frozen snapshot, and results.

For users who start, track, and analyze evaluations. Verification baseline: the project's current implementation as of 2026-10-30. Entry points in deployed environments may vary by version and permissions.

Search keywords: Evaluation Run, draft, task version, run snapshot, copy, cancel.

1. Choose by goal​

2. Find and continue tasks​

  1. Go to Evaluation tasks and find a task by searching its name or filtering by status.
  2. For tasks with incomplete configurations, use Continue configuration, complete the settings, and then start. Saving a draft does not call the evaluation target or scoring model.
  3. For a task that has run, click its name to open Task results, then switch between Results and Settings. Running tasks refresh their status automatically.

3. Understand statuses​

StatusMeaning and action
Not started / DraftThe configuration has not been executed; continue configuring or save a draft.
QueuedSubmitted and waiting for execution resources. Check progress first to avoid duplicate creation.
RunningCalling the evaluation target or scoring.
CancellingA cancellation request has been submitted; waiting for the current execution to settle.
CompletedAll Cases have finished with no failed Cases. This does not imply that business quality requirements are met.
PartialSome calls or scoring attempts failed; completed results remain viewable.
FailedThe execution failed. Check Cases and failure reasons to locate the issue.
CancelledFurther execution has been canceled; completed results are retained.

4. Copy, cancel, and delete​

Using Copy Task in the list immediately creates an independent draft with a default copy name and the configuration retained for review. It does not start automatically. Check the dataset version, evaluation target, evaluator versions, and mappings before and after copying. After the success confirmation, open the configuration to complete it and start.

To stop a running task, click Cancel on the results page and confirm. Cancellation does not erase completed results or reverse external actions the Agent under test has already performed.

Only draft or failed tasks can be deleted from the list. Confirm that you have saved any required results before deleting; the operation cannot be undone. Independent tasks within the same project must have unique names, including drafts. If a name conflicts, change it and continue. You do not need to reenter the entire configuration.

5. Versions and historical facts​

A task binds to specific dataset and evaluator versions, then freezes an execution snapshot at startup. Later asset edits do not rewrite historical inputs, mappings, or scores. When analyzing historical tasks, use the frozen content in Settings. Do not substitute an asset's current content for the configuration actually used in that run.

6. Next steps and related pages​

Return to the manual home page:

AI evaluation platform

Was this page helpful?