Skip to main content

Quick start

Last updated 10/30/2026

Complete your first evaluation with a minimal example, from preparing samples to viewing scores. Start with Preset results, then connect an online Agent once you are familiar with the scoring workflow.

For business users and testers using the platform for the first time. Verification baseline: the project's current implementation as of 2026-10-30. Entry points in deployed environments may vary by version and permissions.

Search keywords: getting started, login, project selection, first evaluation, preset results.

1. Log in and confirm the project​

  1. Open the Agent evaluation entry point in your environment. In a TE SSO environment, follow the page instructions to log in. If the project selection page appears, select a project you have permission to access.
  2. If no projects are accessible, use Switch account or contact an administrator to confirm your authorization. Evaluation business data is not loaded until you select a project.
  3. After entering, you can see Evaluation datasets, Evaluators, Evaluation tasks, and Implementation. Administrators can also see User management.

2. Prepare a minimal evaluation dataset​

The following demonstration data does not represent built-in business rules. Save it as a UTF-8 CSV file, such as first-evaluation.csv.

question,answer,reference_answer
"Can I request a refund within 7 days?","Eligible orders can be refunded within 7 days.","Eligible orders can be refunded within 7 days."
"Can all orders be refunded?","All orders can be refunded unconditionally.","Not all orders can be refunded. Eligibility depends on the order conditions and refund rules."
  1. Go to Evaluation datasets, click Import evaluation dataset, enter a name and upload the example CSV, then click Start importing.
  2. Open the details page, wait for the import to finish, and check the fields and the two samples. Confirm that a published version exists in Version history. Use the actual field keys in subsequent configuration.

3. Prepare a scoring model and evaluator​

  1. First, prepare a callable scoring model under Implementation → Model. If the project already has an available model, reuse it. With TE SSO, AE Agent models automatically use your current login credentials.
  2. Go to Evaluators, click Create, enter a name, create the evaluator, and open its details page.
  3. Select the scoring model, enter the following Prompt, and check the automatically extracted variables. Click Save evaluator and confirm that a published version is generated.
Determine whether the answer to evaluate matches the reference answer.
Question: {{input}}
Answer to evaluate: {{output}}
Reference answer: {{reference_output}}
Scoring: 1 for a complete match, 0.5 for a partial match, and 0 for clear contradictions.
Return only JSON: {"Score": a number from 0 to 1, "Reason": "Explain the basis for the score"}

4. Create and run​

  1. Go to Evaluation tasks → New evaluation task and enter a task name that is not already used in the current project.
  2. Select Preset results as the evaluation target, then select the dataset you just created and its published version.
  3. Select the evaluator you just created and its published version, then complete the mappings: input to the question field, output to the existing answer field, and reference_output to the reference answer field.
  4. Check the sample count and estimated scoring count, then click Confirm and start evaluation. The system scores the existing answers directly without calling an Agent under test.

5. Check completion​

Once the task reaches a terminal state, open Results to view the valid score count, average score, and each Case's Score and Reason. Open Settings to check the run snapshot. Your first evaluation is complete when samples correspond to scoring records, the source of each score can be explained, and the specific failure stage can be located if a failure occurs.

The low-quality answer in the example demonstrates how to locate issues. LLM Judge scores are uncertain, so identical values are not guaranteed on every run. Once familiar with the workflow, switch to an Agent or large language model as the evaluation target. See Create evaluation tasks and map fields for the steps.

6. Next steps and related pages​

Return to the manual home page:

AI evaluation platform

Was this page helpful?