Creating a scorecard for the roleplay agent
A scorecard tells Outdoo AI how to grade a roleplay session. Each roleplay agent can have one scorecard, and every session a team member completes with that agent is scored against it. This article covers the roleplay-specific part: connecting a scorecard to an agent and choosing criteria that suit practice sessions.
To build the scorecard itself, use one of these articles:
- Creating a scorecard: build it manually, objective by objective
- Generating a scorecard with AI: let AI draft it from a prompt
Why use scorecards on roleplay agents?
- Drive consistency in evaluation across every practice session.
- Align roleplay goals with key sales or support behaviors.
- Give team members targeted, specific feedback after each session.
- Track progress over time with measurable scores.
Connect a scorecard to a roleplay agent
- Create or open a roleplay agent from Practice > Roleplay Agents.
- In the agent editor, go to the Scorecard section.
- Open the Select Scorecard dropdown and choose a scorecard. Only published scorecards are listed; drafts do not appear.
- Check the Scorecard Objectives shown below the dropdown to confirm this is the right scorecard for the agent.
- Save the agent.
When you build a scorecard for roleplays, leave Apply to set to Roleplay Agent (the default). The scorecard is then used only for roleplay sessions, and you connect it to agents from the agent editor as described above.
If the scorecard you need does not exist yet
Managers and admins see a Create New Scorecard button under the dropdown. It opens the scorecard editor in a new browser tab. Build and publish the scorecard there, return to the agent editor, and click the refresh icon (Refetch scorecards) next to the Scorecard heading. The new scorecard then appears in the dropdown.
Tips for scoring roleplays
- Score what the agent can test. If the agent's persona never raises a pricing objection, an objective about handling pricing pushback will score poorly for reasons outside the team member's control. Match objectives to the agent's Objections & Priorities.
- Keep it concise: 5 to 10 key behaviors per scorecard.
- Reuse your real-call scorecard where it fits. The same scorecard framework applies to roleplays and real calls, so practice scores and live scores stay comparable.
- Review results after the first few sessions under Performance > Practice, and adjust criteria that score the same way every time.
Hope this article was helpful. For any queries, please write to support@outdoo.ai
Related articles
- Creating a scorecard: build a scorecard manually
- Generating a scorecard with AI: let AI draft the scorecard from a prompt
- Create a scorecard using AI (with resources): ground the AI draft in your own document
- Scorecards with attached resources: attach a reference document to an existing scorecard