Strategy Selection Failures

The Problem

Agent chooses suboptimal RAG strategy (Redwood vs Cedar vs Cypress) for query, leading to poor results or inefficient processing.

Symptoms

• ❌ Simple query uses complex Cypress (slow + expensive)
• ❌ Complex query uses basic Redwood (incomplete answer)
• ❌ Strategy selection inconsistent
• ❌ Cannot predict which strategy agent will use
• ❌ Wrong strategy for user context

Real-World Example

Query: "What is the API rate limit?" (Simple factual)

Expected: Redwood (standard RAG, fast, cheap)
→ Retrieve chunks
→ Answer directly

Agent chose: Cypress (advanced agentic, slow, expensive)
→ Planning phase
→ Multi-step reasoning
→ Tool calls
→ Same answer, 5x latency, 3x cost

Strategy overkill for simple query

---

Deep Technical Analysis

Strategy Characteristics

Redwood (Standard RAG):

Best for:
→ Simple factual queries
→ Single-hop reasoning
→ Well-documented topics

Example: "What is X?"
→ Fast: 1-2 seconds
→ Cost: $0.01/query

Cedar (Context-Aware):

Best for:
→ User-specific queries
→ Permission-based retrieval
→ Personalized answers

Example: "What can I access?"
→ Needs user context
→ Moderate: 2-4 seconds
→ Cost: $0.02/query

Cypress (Advanced Agentic):

Best for:
→ Complex multi-step
→ Requires planning
→ Tool orchestration

Example: "Compare X, Y, Z and recommend"
→ Slow: 5-15 seconds
→ Cost: $0.05-0.10/query

Strategy Selection Logic

Query Complexity Detection:

Features:
→ Query length (tokens)
→ Question type (what, how, compare)
→ Named entities count
→ Conditional clauses

Simple: "What is X?" → Redwood
Medium: "How does X work with Y?" → Cedar
Complex: "If A then compare B vs C, considering D" → Cypress

Context Requirements:

Check if query needs:
→ User permissions: Use Cedar
→ Conversation history: Use Cedar
→ External tools: Use Cypress

Keywords:
→ "I", "my", "our" → User-specific → Cedar
→ "compare", "analyze", "recommend" → Complex → Cypress

Failure Modes

Under-Strategizing:

Complex query: "Explain how authentication, authorization, and audit
logging interact in our system"

Agent uses: Redwood (simple)
→ Retrieves authentication docs
→ Gives partial answer
→ Misses interaction analysis

Should use: Cypress
→ Break into sub-queries
→ Retrieve for each component
→ Synthesize relationships

Over-Strategizing:

Simple query: "API rate limit?"

Agent uses: Cypress
→ Planning: "Let me break this down..."
→ Sub-query 1: Rate limits
→ Sub-query 2: API documentation
→ Synthesis: Combine findings
→ 10 seconds

Redwood sufficient:
→ One retrieval
→ Direct answer
→ 2 seconds

Optimization Strategies

Classification Model:

Train classifier:
→ Input: Query text
→ Output: Strategy (Redwood / Cedar / Cypress)

Training data:
→ 1000 queries labeled by experts
→ Features: Length, complexity, keywords

Accuracy target: >85%

Rule-Based Heuristics:

Rules:
1. If query < 20 tokens AND simple question word → Redwood
2. If query contains "my", "I", "our" → Cedar
3. If query contains "compare", "analyze", "steps" → Cypress
4. Default: Redwood

Fast, interpretable

Adaptive Selection:

Start with Redwood:
→ If insufficient (confidence < 0.70) → Escalate to Cedar
→ If still insufficient → Escalate to Cypress

Progressive complexity
Minimize cost while ensuring quality

---

How to Solve

Implement query complexity classifier (ML or rule-based) + detect user-context requirements (keywords: my, I, our) + use Redwood by default, escalate if needed + monitor strategy selection accuracy + measure cost/latency per strategy + test strategy assignment on eval set + provide strategy override for power users + alert on frequent strategy escalations (indicates classifier issue). See Strategy Selection.

---

Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the ask query parameter:

GET /dev/rag-scenarios-and-solutions/agent/strategy-selection.md?ask=<question>

The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.