Skip to main content

View results and execution traces

Last updated 10/30/2026

Start with overall metrics, then drill into Case details to distinguish answer quality, execution failures, and scoring failures. Use the frozen configuration to interpret the evaluation.

For users who need to analyze results, export low-scoring samples, and troubleshoot issues. Verification baseline: the project's current implementation as of 2026-10-30. Entry points in deployed environments may vary by version and permissions.

Search keywords: average score, valid scores, pass rate, low scores, export, failure retries, Trace.

1. Check valid scores and failure counts first​

  1. Open a task and go to Results. First, check the task status, Case execution progress, and valid score count.
  2. Review the average score, score distribution, and each evaluator's success, failure, and cancellation counts before deciding whether there are enough results to support a conclusion.
  3. Switch to Settings to check frozen content such as the dataset, evaluation target, evaluator versions, random sampling settings, and field mappings.

Average scores and distributions include only valid Scores obtained successfully. Failures and empty scores are not counted as 0. A Case evaluated by multiple evaluators produces multiple scores, so the scoring count is not the sample count. For example, two Cases and two evaluators produce four scores. If three are valid, "3/2" cannot be used to explain the success rate.

If the task has a pass threshold, the pass rate means "valid scores meeting the threshold ÷ valid scores," with failures and cancellations listed separately. If there is no threshold or no valid score, do not treat a blank metric as 0%. This metric provides quality information only and does not automatically determine release eligibility.

2. Find low-scoring or failed Cases​

  1. Use Score no higher than to set the low-score range, then select Lowest score first to find the Cases most in need of review.
  2. To narrow the results, open More filters and combine evaluator, scoring status, Case status, execution evidence completeness, and keywords.
  3. If the task has frozen dynamic filter fields, you can filter by exact text values, numeric ranges, Boolean values, missing fields, or NULL. Fields that were not frozen remain viewable in the details but may not support list filtering.
  4. Click a Case to open the details drawer and view evaluation content, Judge scores, Reason, inputs, outputs, and failure reasons. View the raw data snapshot if needed.

A successful Agent call does not imply successful Judge scoring. A Judge parsing failure does not mean the answer quality is zero. First distinguish the target's execution stage from the scoring stage, then decide whether to change the integration, data, or scoring rules.

3. Read execution traces​

  1. When a trace is available for an online Agent Case, check the collection status, completeness, and step tree in the execution trace section of its details.
  2. Expand problematic steps and check their status and redacted input/output previews. When duration, Token, or cost data is available, interpret it according to the source's coverage.
  3. If only an external reference is available without step details, follow the safe link to the source system. Partial, truncated, or unfinished collection must not be treated as a complete execution record.

Current AE Agent / TE Claude traces mainly cover recorded tool and sub-Agent calls, parent-child relationships, and redacted previews. They do not imply coverage of complete model reasoning, internal steps of remote delegation, or all duration and cost data. Direct model calls and preset results have no actual Agent tool-call sequence.

Final answers can be scored without traces. Process evaluators that explicitly depend on execution_trace or tool_calls require available, complete evidence. If the evidence does not meet these requirements, only that score fails and remains empty. Successful collection after a retry does not automatically rewrite existing historical scores.

4. Retry failed Cases​

  1. When a task is Failed or Partial and has Cases that failed at the Agent stage, use Retry failed Cases.
  2. Read the confirmation instructions: only eligible Cases with Agent-stage statuses such as FAILED, BLOCKED, or UNKNOWN are retried. Successful Cases are not rerun.
  3. Retries reuse the current Run's frozen snapshot. New results replace the current results of those failed Cases, and aggregates are recalculated. Continue checking statuses and failure reasons after retrying.

This entry point does not arbitrarily rerun all low-scoring answers or items that failed only at the Judge stage. If you changed the data, model, or rules and want to validate the new configuration, copy the task, check versions and mappings, and start a new evaluation.

5. Copy and export​

  1. Set the filters and maximum score as needed, and confirm that the list matches the scope you want to export.
  2. Click Export, select CSV or JSONL for Current filtered results, and wait for the background process to generate the file for download.
  3. For a single Case, use Copy structured JSON. If the browser blocks automatic copying, follow the page instructions to select and copy the text manually.

In this version, low-scoring Cases are exported through the current filtered results by default; there is no separate low-score export menu. Standard copy and export do not include Trace step details, external Trace URLs, Provider Request ID, or tool arguments and results. View this information in authorized details pages and source systems when needed.

6. Next steps and related pages​

Return to the manual home page:

AI evaluation platform

Was this page helpful?