Kaia Coach Card Writing Best Practices

Created by Mike Goodwin, Modified on Mon, 13 Apr at 10:54 AM by Aye Myat

Objective

Learn best practices for writing coach card questions that produce accurate and consistent AI scoring results in Outreach. This guide covers general principles for all question types as well as type-specific tips for rating scales, yes/no, open-ended, multiple choice, and single choice questions.

Applies To

Outreach Users
Kaia

Before You Begin

Before enabling AI scoring on a question, verify:

Question can be answered from call transcript only
All company terms and acronyms are defined
For rating scales: Every score level has clear criteria
For yes/no: Both "Yes" and "No" conditions are defined
For multiple choice: Each choice has distinct, specific definition
For single choice: All realistic scenarios are covered
For open questions: Desired format and scope are specified
No compound questions (split if needed)

Best Practices

General Best Practices (All Question Types)

Best Practice	Why It Matters	Example
Base questions on the call transcript	AI can only analyze what was said during the call. Avoid asking about actions taken after the call or information not discussed.	❌ "Did the rep send a follow-up email?" ✅ "Did the rep confirm they would send a follow-up email?"
Write for an external observer	The AI doesn't know your company-specific terms, abbreviations, or internal processes. Define everything clearly.	❌ "Did the rep use our QPR framework?" ✅ "Did the rep use our Qualify-Position-Resolve framework (asking qualifying questions, positioning value, then resolving objections)?"
Avoid compound questions	Questions with multiple parts confuse the AI and reduce accuracy. Split into separate questions.	❌ "Did the rep establish rapport and identify pain points?" ✅ Question 1: "Did the rep establish rapport?" ✅ Question 2: "Did the rep identify pain points?"
Define ambiguous terms	If a term could have multiple meanings, explain exactly what you mean.	❌ "Did the rep identify the champion?" ✅ "Did the rep identify the champion (the person who will advocate for our solution internally)?"

Question Type-Specific Tips

Rating Scale Questions

Tip	Details	Example
Define EVERY score option	Provide clear criteria for each point on the scale. AI needs to know the difference between a 3 and a 4.	Question: "Rate the rep's objection handling" Score Definitions: 1 - Rep avoided or dismissed objections without addressing them 2 - Rep acknowledged objections but gave weak or generic responses 3 - Rep addressed objections with relevant information but missed some key points 4 - Rep thoroughly addressed objections with specific examples and value propositions 5 - Rep proactively surfaced and resolved objections before they became blockers, using compelling evidence
Make criteria observable, not interpretive	Base scoring criteria on specific behaviors visible in the transcript, not on tone or feeling.	❌ "5 = Rep seemed very knowledgeable" ✅ "5 = Rep cited 3+ specific product features with technical details and customer use cases"

Complete Rating Scale Example:

Question: "Rate how well the rep established next steps"
Score Definitions:
1 - No next steps discussed; call ended without clarity on what happens next
2 - Vague next steps mentioned (e.g., "I'll follow up soon") without specifics
3 - Specific next step identified (e.g., "Demo on Tuesday") but without confirmed commitment
4 - Specific next step with confirmed date/time and identified action owner
5 - Specific next step with confirmed date/time, action owner, AND contingency plan if timing changes

Yes/No Questions

Tip	Details	Example
Define what qualifies as "Yes"	Be extremely specific about what must happen for the answer to be "Yes". AI has a bias toward "Yes" so set a high bar.	Question: "Did the rep identify the decision-making process?" "Yes" means: Rep explicitly asked about and the prospect described the steps, timeline, and people involved in making the purchase decision. Mentioning "I'll need to check with my team" does NOT count as "Yes".
Define what qualifies as "No"	Also specify when the answer should be "No" to avoid ambiguity.	"No" means: Rep did not ask about the decision process, OR prospect did not provide specific details about the process.
Specify minimum requirements	When asking if something happened, clarify how many times or to what extent.	❌ "Did the rep ask discovery questions?" ✅ "Did the rep ask at least 5 discovery questions about the prospect's current situation and challenges?"

Complete Yes/No Example:

Question: "Did the rep multi-thread by engaging multiple stakeholders?"
Additional Instructions:
"Yes" means: The rep either (1) had 2+ people from the prospect's company on this call, OR (2) explicitly mentioned speaking with other stakeholders and named them by role/name (e.g., "I talked to your CFO Sarah yesterday").
"No" means: Only one prospect on the call AND rep did not mention engaging other stakeholders, OR rep made vague reference like "I'll need to talk to others" without specifics.

Open Questions

Tip	Details	Example
Use "How", "Why", and "What" for comprehensive answers	These question starters prompt AI to provide detailed, thoughtful responses.	✅ "How did the rep handle pricing objections?" ✅ "Why did the prospect express interest in our solution?" ✅ "What specific challenges did the prospect mention?"
Specify the format you want	Tell AI whether you want a summary, bullet points, direct quotes, or analysis.	"Summarize the top 3 pain points mentioned by the prospect. For each pain point, provide a 1-2 sentence description."
Limit scope to avoid overly broad answers	Narrow the focus so AI provides actionable insights rather than generic summaries.	❌ "What happened on the call?" (too broad) ✅ "What specific budget constraints did the prospect mention?"

Complete Open Question Example:

Question: "What were the prospect's top priorities for solving their current challenges?"
Additional Instructions:
Provide a 3-5 sentence summary of the prospect's stated priorities. For each priority, include:
1. What the priority is (e.g., "Reduce manual data entry")
2. Why it matters to them (e.g., "Team spending 10 hours/week on this")
If the prospect did not clearly state priorities, respond with "Priorities not clearly identified during this call."

Multiple Choice Questions (Select All That Apply)

Tip	Details	Example
Define each choice clearly and distinctly	Make each option specific enough that AI can distinguish between similar choices. Include what qualifies and what doesn't.	Question: "Which MEDDIC elements were discussed? (Select all that apply)" Choices: ☐ Metrics - Quantified business impact or ROI was discussed (e.g., "save 20 hours/week" or "increase revenue by 15%") ☐ Economic Buyer - Person with budget authority was identified by name or role ☐ Decision Criteria - How they'll evaluate solutions was explicitly discussed ☐ Decision Process - Steps and timeline for making decision were outlined ☐ Identify Pain - Specific business problems or challenges were identified ☐ Champion - Internal advocate who will sell for us was identified
Provide clear criteria for each choice	Don't just list options — define what must be true for each option to be selected.	Choice: "Pricing objection" Criteria: Prospect expressed concern about cost, pricing being too high, or needing budget approval. Does NOT include general budget discussion without concern.
Include an "Other" or "None" option when relevant	This prevents AI from forcing a selection when nothing applies.	Choices: Budget discussed, Timeline discussed, Competition discussed, None of these

Complete Multiple Choice Example:

Question: "Which stakeholder concerns were raised during the call? (Select all that apply)"
Choices and Definitions:
☐ Security/Compliance Concerns
Definition: Prospect asked about data security, compliance certifications (SOC2, GDPR, etc.), or data handling practices.
☐ Integration Complexity
Definition: Prospect expressed concern about technical integration effort, API availability, or compatibility with existing systems.
☐ ROI/Pricing Justification
Definition: Prospect asked for proof of value, customer case studies, or expressed need to justify cost to leadership.
☐ Implementation Timeline
Definition: Prospect concerned about how long implementation takes or whether it fits their timeline.
☐ None of these concerns were raised

Single Choice Questions (Select Only One)

Tip	Details	Example
Make each choice comprehensive and distinct	Each option should represent a complete, mutually exclusive scenario. Define boundaries clearly.	Question: "What was the primary call objective achieved?" Choices: ○ Discovery Completed - Rep successfully uncovered prospect's needs, challenges, budget, and timeline ○ Demo Delivered - Rep presented solution and demonstrated key features relevant to prospect's needs ○ Next Steps Secured - Rep scheduled specific next meeting or commitment from prospect ○ Relationship Building - Call focused on rapport and initial qualification, no specific commitment yet ○ No Clear Objective Achieved - Call ended without accomplishing a defined sales objective
Cover all realistic possibilities	Ensure your choices account for all scenarios that might occur. Include a catch-all if needed.	Include options like "Not discussed", "Unclear from call", or "Other" to handle edge cases
Define what breaks ties	If multiple things could apply, tell AI which takes priority.	Tie-breaker: "If multiple objections were raised with equal emphasis, select the one discussed first in the call."

Complete Single Choice Example:

Question: "How well did the rep control the call flow? (Select one)"
Choices and Definitions:
○ Led the Conversation
Definition: Rep set clear agenda at start, guided discussion with structured questions, redirected when prospect went off-topic, and maintained control of timing and topics throughout.
○ Collaborative Flow
Definition: Rep and prospect both contributed to directing the conversation. Rep asked questions but also followed prospect's lead on important topics. Balanced give-and-take.
○ Followed Prospect's Lead
Definition: Prospect controlled most of the conversation direction. Rep primarily responded to prospect's questions and topics rather than guiding with their own agenda.
○ Lost Control
Definition: Conversation was disorganized or went significantly off-track. Rep struggled to redirect, missed key qualification topics, or call ended without covering intended agenda.
○ Unable to Determine
Definition: Call context insufficient to assess who controlled the flow, or call type doesn't require rep to control (e.g., customer success check-in).
Tie-breaker Rule: If control shifted during the call, select based on who controlled the majority of the call time (more than 50%).

Common Pitfalls to Avoid

Pitfall	Why It's a Problem	How to Fix It
Using jargon without definition	AI may not know your company's internal terms or industry acronyms.	Define every acronym and company-specific term: "Did the rep identify BANT? (Budget, Authority, Need, Timeline)"
Asking about post-call actions	AI only has access to the call transcript, not what happened after.	❌ "Did the rep update CRM?" ✅ "Did the rep confirm they would update the opportunity details in CRM?"
Assuming AI knows your methodology	AI doesn't inherently know MEDDIC, Challenger Sale, SPIN, etc. without explanation.	Always define frameworks: "Using SPIN Selling (Situation, Problem, Implication, Need-payoff), rate..."
Vague quantifiers	Terms like "several", "many", "good" are subjective.	❌ "Did rep ask many questions?" ✅ "Did rep ask at least 8 questions?"
Asking about tone or emotion	AI analyzes text transcript, not voice tone or body language.	❌ "Did the prospect seem excited?" ✅ "Did the prospect use positive language like 'This is exactly what we need' or 'I'm excited about this'?"