---
title: "Creating a scorecard"
url: "https://help.outdoo.ai/en/article/creating-scorecard"
language: "en"
last_updated: "2026-10-23T02:35:56.869Z"
---

# Creating a scorecard

Learn how to create scorecards for evaluating conversations. Discover whether to use templates, customize from scratch, or generate one with AI.

A scorecard is a set of criteria used to evaluate a conversation. It tells reviewers, and the AI, what to look for and how to score what they observe. The same scorecard framework applies to both real calls and roleplay sessions, so practice scores and real-call scores are directly comparable.

This article covers building a scorecard **manually**, objective by objective. If you would rather have Outdoo AI draft one for you, see [Generating a scorecard with AI](https://help.outdoo.ai/en/article/generating-a-scorecard-with-ai).

## What a scorecard contains

-   **Scorecard details:** a title, a final objective (what the scorecard is for), the AI scoring mode, what the scorecard applies to, the scorecard model, and optional reference resources.

-   **Objectives:** the individual behaviors being evaluated, for example "Confirmed budget" or "Asked about timeline". Each objective has its own scoring criteria, split into **Must do** (points required for a passing score) and **Good to do** (points that go above and beyond), and can be tagged with skills. On Caliber 2 scorecards, each objective also has a **Format** and a **When to mark N/A** note.

-   **Overall score:** how the objectives combine into one score, either with equal weights or with a weight per objective.

Keep scorecards focused: 5 to 10 objectives is usually enough. More than that and reviewers start skimming.

![Scorecards settings page listing published and draft scorecards](https://cdn.prod.website-files.com/666fdec617b81d256ca8f1ca/6ada4af8a56b74237e80d2ac_oct22-scorecards-list.avif)

*The Scorecards page, where every scorecard you create appears alongside its publish status.*

## Create a scorecard manually

-   Go to [**Manage > Scorecards**](https://app.outdoo.ai/manage/scorecards) and click **Add Scorecard**. The **Create a new scorecard** window opens.

-   Click **Start from scratch** to begin with an empty scorecard, or click **Select** on a template that is close to your use case. A template pre-fills a set of objectives that you can edit or delete. To have Outdoo AI draft the scorecard instead, use **Build with AI** (see [Generating a scorecard with AI](https://help.outdoo.ai/en/article/generating-a-scorecard-with-ai)).

-   On **Scorecard Details**, enter the **Scorecard Title** and **Final Objective**, and pick an **AI Scoring Mode** (Standard, Strict, or Very Strict). What each mode does depends on the scorecard model. See "AI Scoring Mode by scorecard model" below.

-   Under **Apply to**, choose where the scorecard is used: **Roleplay Agent** (the default), **Team**, **Team Member**, **Category**, or **Deal Stage**. Choose **Roleplay Agent** to use the scorecard only for roleplay sessions.

-   Under **Scorecard model**, choose **Caliber 1** or **Caliber 2**. New scorecards start on **Caliber 2**, which is marked **Recommended**. See "Choose a scorecard model" below.

-   Optionally, set **Recommendation Context** (which data the AI draws on for coaching recommendations) and attach a reference document under **Resources**. See [Scorecards with attached resources](https://help.outdoo.ai/en/article/scorecards-with-attached-resources).

-   In the left panel, under **Objectives**, select an objective to edit it, or click **+** to add one. For each objective, write what the team member should demonstrate, fill in **Must do** and **Good to do**, and add skills if relevant. On Caliber 2, also set **Format** and **When to mark N/A**. See "Set up each objective" below. Use **Prev** and **Next** to move between objectives.

-   Select **Overall Score** in the left panel and choose a **Score Methodology**: **Equal Distribution** (every objective counts the same) or **Weighted Distribution** (set a weight from 1 to 10 for each objective).

-   Click **Publish** to make the scorecard available, or **Save Draft** to finish it later.

Templates are fully editable. Change the objectives, criteria, or weights at any point, including after publishing.

To use this scorecard in practice sessions, select it on a roleplay agent. See [Creating a scorecard for the roleplay agent](https://help.outdoo.ai/en/article/creating-scorecard-for-roleplay-buyer-bot).

## Choose a scorecard model

The **Scorecard model** setting on **Scorecard Details** controls how the scorecard is evaluated.

![Scorecard model dropdown with Caliber 1 and Caliber 2 (Recommended)](https://cdn.prod.website-files.com/666fdec617b81d256ca8f1ca/6ada4af408f67ddd8aeef18e_oct22-scorecard-model-dropdown.avif)

*Pick Caliber 1 or Caliber 2 in Scorecard model, at the bottom of Scorecard Details.*

![Animation: choosing Caliber 2, opening an objective, adding a Must do point and picking a Format](https://cdn.prod.website-files.com/666fdec617b81d256ca8f1ca/6ada4b042014ddf96a594c28_oct22-scorecard-caliber2-setup.gif)

*Setting up a Caliber 2 objective.*

| Model | How range scores work | When a question is N/A |
| --- | --- | --- |
| **Caliber 1** | The AI picks each range score. It can return any number, capped at 5. | The AI marks a question N/A when it decides the question does not apply. |
| **Caliber 2** (Recommended) | Scores are calculated from the **Must do** and **Good to do** checklist, in half-point steps from 0 to 5. | A question is N/A when it has no checklist items to evaluate. You can also describe when it should be marked N/A. |

New scorecards, including ones generated with AI, use **Caliber 2**. Scorecards created before the scorecard model setting existed open as **Caliber 1**. You can switch an existing scorecard at any time.

**Switching to Caliber 1.** Caliber 1 does not use a custom format or a when-to-mark-N/A note. If any objective has a Yes/No format or a **When to mark N/A** note, the **Switch to Caliber 1?** window lists how many objectives will return to the Range format and how many will have their N/A note cleared. Objective text, scoring criteria, and skills stay as they are. Click **Switch to Caliber 1** to confirm.

![Switch to Caliber 1 window listing the question settings that will reset](https://cdn.prod.website-files.com/666fdec617b81d256ca8f1ca/6ada4af68e71f53d96e4d173_oct22-scorecard-switch-caliber1.avif)

*The Switch to Caliber 1? window tells you what will be reset before you confirm.*

**Switching to Caliber 2.** If you try to edit **Format** or **When to mark N/A** on a Caliber 1 scorecard, the **Switch to Caliber 2** window opens. Click **Switch** to change the scorecard model. You stay on the same objective.

## AI Scoring Mode by scorecard model

The three modes have the same names on both models, but they work differently.

| Mode | Caliber 1 | Caliber 2 |
| --- | --- | --- |
| **Standard** | The AI picks a score with some tolerance and gives partial credit when coaching points are partly followed. | Must do items count for 70% of the score and Good to do items for 30%. A partial mark counts as half. With no Good to do items, the score comes only from Must do items. |
| **Strict** | More demanding than Standard. Coaching points need to be clearly followed for a high score, with some flexibility in phrasing. | More demanding than Standard. Missed Must do items pull the score down further. Good to do items still add credit once Must do items are largely met. |
| **Very Strict** | A high score only when coaching points are followed exactly. Best for compliance-heavy teams or high-stakes calls. | Every Must do item has to be met before the score can go above 4. All Must do items met but Good to do items missed scores 4. The score is 5 only when every checklist item is met. |

On Caliber 2, each Must do and Good to do item is marked met, partial, or not met, and scores land on half-point steps from 0 to 5.

## Set up each objective

-   **Scoring Criteria (Must do and Good to do):** on Caliber 2 scorecards, each is a list of separate points. Type a point and press Enter, or click **Add point**, to start the next one. Click **Remove point** to delete one. Write one observable behavior per point, because each point is marked on its own. On Caliber 1 scorecards, Must do and Good to do are free-text boxes.

-   **Format (Caliber 2 only):** **Range (0 to 5)** scores the objective from 0 to 5. **Yes (5)/ No (0)** scores Yes as 5 and No as 0.

-   **When to mark N/A (Caliber 2 only):** describe when the objective should be marked N/A and left out of the score, for example "Mark N/A if the customer never raises an objection."

-   **Skills:** tag up to 10 skills per objective.

![Caliber 2 objective editor with Must do and Good to do points and add buttons](https://cdn.prod.website-files.com/666fdec617b81d256ca8f1ca/6ada4aed234014369825938a_oct22-scorecard-caliber2-objective.avif)

*On Caliber 2, each Must do and Good to do item is its own point.*

![Format options Range (0 to 5) and Yes (5)/ No (0), and a When to mark N/A note](https://cdn.prod.website-files.com/666fdec617b81d256ca8f1ca/6ada4af045a4f5bec244369c_oct22-scorecard-format-na.avif)

*Choose a Format and describe when the objective should be marked N/A.*

## Tips

-   Use clear, observable language. "Team member asked at least two open-ended discovery questions" is easier to score consistently than "Team member showed curiosity."

-   Align each objective with a real behavior you want team members to change or reinforce.

-   Review the scorecard after the first few uses. Criteria that score the same way on every call are either too easy or too hard and should be adjusted.

Hope this article was helpful. For any queries, please write to support@outdoo.ai

## Related articles

-   [Generating a scorecard with AI](https://help.outdoo.ai/en/article/generating-a-scorecard-with-ai): let AI draft the scorecard from a prompt

-   [Create a scorecard using AI (with resources)](https://help.outdoo.ai/en/article/create-a-scorecard-using-ai-with-resources): ground an AI-generated scorecard in your own document

-   [Scorecards with attached resources](https://help.outdoo.ai/en/article/scorecards-with-attached-resources): attach a reference document to an existing scorecard

-   [Creating a scorecard for the roleplay agent](https://help.outdoo.ai/en/article/creating-scorecard-for-roleplay-buyer-bot): use a scorecard to grade roleplay sessions