Agents
Testing in Playground

Testing in Playground

The Playground is your environment for testing and refining agents before deployment.

Accessing the Playground

  1. Navigate to Playground in the sidebar
  2. Select an agent from the dropdown
  3. Start a conversation

What to Test

Basic Functionality

  • Can the agent answer questions from your documents?
  • Are responses accurate and relevant?
  • Does the tone match your expectations?

Edge Cases

Test scenarios that might trip up your agent:

  • Questions not covered by your documents
  • Ambiguous or unclear questions
  • Multi-part questions
  • Follow-up questions requiring context

Conversation Flow

Test multi-turn conversations:

  • Does the agent remember context from earlier messages?
  • Can it handle topic changes gracefully?
  • Does it ask for clarification when needed?

Evaluating Responses

For each response, consider:

AspectQuestions to Ask
AccuracyIs the information correct?
RelevanceDoes it answer the actual question?
CompletenessIs anything important missing?
ToneDoes it match your brand voice?
LengthIs it appropriately concise or detailed?

Iterative Improvement

Identify Issues

Common problems and solutions:

IssueSolution
Wrong informationCheck document content, improve retrieval
Too verboseAdd "be concise" to system prompt
Too formal/casualAdjust tone in system prompt
Doesn't know answerAdd relevant documents
Makes up factsStrengthen boundaries in prompt

Make Changes

  1. Identify the issue in Playground testing
  2. Adjust relevant settings (prompt, documents, configuration)
  3. Test again with the same questions
  4. Verify the fix doesn't break other behaviors

Test Scenarios Checklist

Create a set of standard test questions:

## Test Scenarios for [Agent Name]
 
### Basic Questions
- [ ] Q: "What is [product]?" — Expected: Clear description
- [ ] Q: "How much does it cost?" — Expected: Pricing info
 
### Edge Cases
- [ ] Q: Random unrelated question — Expected: Polite redirect
- [ ] Q: Competitor comparison — Expected: Stays on topic
 
### Follow-ups
- [ ] Q: Initial question, then "Tell me more" — Expected: Expands on previous answer

Save your test scenarios and re-run them after making changes to catch regressions.

Ready for Deployment

Your agent is ready when:

  • ✅ Answers common questions accurately
  • ✅ Handles edge cases gracefully
  • ✅ Maintains appropriate tone
  • ✅ Respects defined boundaries
  • ✅ Multi-turn conversations work well

Next, deploy your agent to channels where users will interact with it.