Inbox & Training

The Inbox is your quality control center where you review, edit, and improve AI responses to continuously train and optimize your agents.

Overview

The Inbox displays all AI interactions for review, enabling you to:

• Monitor response quality
• Correct inaccurate answers
• Identify knowledge gaps
• Train agents through feedback
• Create knowledge base articles

Accessing the Inbox

Navigate to Inbox from the main menu to see recent interactions.

Interaction List

Each interaction shows:

• Question: What the user asked
• Response: AI's answer
• Agent: Which agent handled it
• Sources: Citations used
• Status: Review status
• Timestamp: When it occurred
• Rating: User feedback (if provided)

Review Statuses

Status	Icon	Description
Not Reviewed	⚪	Awaiting review
Accurate	✅	Confirmed correct
Edited	✏️	Response modified
Inaccurate	❌	Flagged as wrong
No Answer	⚠️	Agent couldn't respond

Reviewing Interactions

Mark as Accurate

When response is correct:

1. Click on interaction
2. Review response and sources
3. Click Mark as Accurate ✅
4. Agent learns this response was good

Edit Response

When response needs improvement:

1. Click on interaction
2. Click Edit Response
3. Make corrections
4. Click Save
5. Choose action:
   • Update Only: Save for records
   • Add to Training: Agent learns from edit
   • Create KB Article: Convert to knowledge base

Example:

Original Response:
"The Pro plan costs $99/month."

Edited Response:
"The Pro plan costs $99/month for up to 50 users, with
$2 per additional user. Enterprise plans with 100+ users
get custom pricing."

Action: Add to Training ✓

Mark as Inaccurate

When response is wrong:

1. Click interaction
2. Click Mark as Inaccurate ❌
3. Provide reason:
   • Wrong information
   • Poor sources
   • Hallucination
   • Off-topic
4. Optionally provide correct answer
5. Save

This flags the interaction for investigation and improvement.

Filtering & Search

Filter Options

By Status:

• Not Reviewed
• Accurate
• Edited
• Inaccurate
• No Answer

By Agent:

• Select specific agent
• Compare agent performance

By Date:

• Today
• Last 7 days
• Last 30 days
• Custom range

By Source:

• Web
• Browser Extension
• Widget
• API
• Integration

By Rating:

• 5 stars
• 4 stars
• 3 stars or below
• No rating

Search

Search interactions by:

• Keywords in questions
• Keywords in responses
• Topic or category
• User ID

Example:

Search: "pricing"
Results: All interactions about pricing

Search: "error" in responses
Results: All responses mentioning errors

Bulk Operations

Bulk Review

1. Select multiple interactions (checkbox)
2. Choose action:
   • Mark as Accurate
   • Mark as Inaccurate
   • Export to CSV
   • Add to eval set
3. Confirm

Bulk Export

Export interactions for analysis:

Format options:

• CSV (spreadsheet analysis)
• JSON (programmatic processing)
• PDF (reporting)

Fields included:

• Question, Response, Agent, Sources
• Timestamp, Status, Rating
• Metadata (tokens, latency, etc.)

Training Your Agent

Learning from Edits

When you edit responses:

1. Edit saved to database
2. Original + edited pair stored
3. Agent learns from comparison
4. Future similar queries improved

Example Training Cycle:

Query: "How do I reset my password?"

Original Response (v1):
"You can reset your password in the settings."

Edited Response:
"To reset your password:
1. Go to Settings > Account
2. Click 'Change Password'
3. Enter current password
4. Enter new password (min 8 characters)
5. Click Save"

Next Similar Query:
Agent now provides detailed steps ✅

Creating KB Articles

Convert high-quality interactions to knowledge base articles:

1. Find accurate, well-cited interaction
2. Click Create KB Article
3. Review and enhance:
   • Add title
   • Organize content
   • Add tags
   • Format nicely
4. Publish to KB

When to Create KB Articles: ✅ Frequently asked questions ✅ High-quality responses ✅ Complete, accurate information ✅ Well-cited sources ❌ One-off, specific questions ❌ Time-sensitive information

Identifying Knowledge Gaps

Look for patterns in "No Answer" interactions:

Common "No Answer" Topics:
1. API v2 endpoints (15 instances)
   → Action: Add API v2 documentation

2. New feature "Project Templates" (8 instances)
   → Action: Create feature documentation

3. Pricing for non-profits (5 instances)
   → Action: Add non-profit pricing info

Analytics from Inbox

Quality Metrics

Track response quality:

• Accuracy Rate: % marked accurate
• Edit Rate: % requiring edits
• No Answer Rate: % with no response
• User Satisfaction: Average rating

Agent Comparison

Compare agents side-by-side:

Metric	Agent A	Agent B
Accuracy	92%	87%
Avg Response Time	1.8s	2.3s
User Rating	4.5	4.2
No Answer Rate	5%	8%

Topic Analysis

Identify common topics:

• Pricing (1,234 interactions)
• Technical Support (892)
• Billing (456)
• Features (332)

Best Practices

1. Regular Reviews

✅ Review inbox daily or weekly ✅ Prioritize unreviewed interactions ✅ Focus on highly-rated for KB ✅ Investigate low-rated thoroughly ❌ Don't let backlog build up

2. Consistent Feedback

✅ Use consistent criteria for "accurate" ✅ Provide specific edit reasons ✅ Document improvement patterns ❌ Don't have multiple reviewers with different standards

3. Pattern Recognition

✅ Look for repeated issues ✅ Identify knowledge gaps ✅ Track trending topics ✅ Monitor seasonal patterns ❌ Don't review in isolation

4. Actionable Improvements

✅ Fix identified knowledge gaps ✅ Update agent instructions based on patterns ✅ Add missing data sources ✅ Create KB articles from good responses ❌ Don't just mark without improving

Automation

Auto-Training

Configure automatic training:

{
  "autoTraining": {
    "enabled": true,
    "threshold": 4.5,        // Min rating for auto-training
    "requireCitation": true, // Must have sources
    "reviewRequired": false  // Skip manual review
  }
}

Auto-KB Generation

Automatically create KB articles:

{
  "autoKB": {
    "enabled": true,
    "minRating": 5,          // Only 5-star interactions
    "minOccurrence": 3,      // Asked at least 3 times
    "requireReview": true    // Admin reviews before publishing
  }
}

Troubleshooting

Inbox Not Updating

Check:

1. Interactions are being created
2. Filters aren't too restrictive
3. Date range includes recent activity
4. Agent filter includes active agents

Can't Edit Responses

Check:

1. User has edit permissions (Manager+)
2. Interaction isn't locked
3. Agent still exists
4. Browser has connectivity

Training Not Working

Check:

1. Training feature is enabled
2. Sufficient edit examples provided
3. Allow time for learning (not instant)
4. Verify similar queries improved

Next Steps

• Analytics Dashboard - View metrics
• Evaluation Framework - Automated testing
• Performance Tuning - Optimize agents
• Knowledge Base - Manage KB articles

---

Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the ask query parameter:

GET /dev/product/monitoring/inbox-training.md?ask=<question>

The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.