## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# 🧠 Multi-IA Router

## Routing Inteligente entre Múltiples Proveedores de LLM

Version: 0.1.0 Status: Specification (VAPORA v1.0 - Multi-Agent Multi-IA) Purpose: Sistema de routing dinámico que selecciona el LLM óptimo por contexto

## 🎯 Objetivo

Problema:

- Cada tarea necesita un LLM diferente (code ≠ embeddings ≠ review)
- Costos varían enormemente (Ollama gratis vs Claude Opus $$$)
- Disponibilidad varía (rate limits, latencia)
- Necesidad de fallback automático

Solución: Sistema inteligente de routing que decide qué LLM usar según:

1. Contexto de la tarea (type, domain, complexity)
2. Reglas predefinidas (mappings estáticos)
3. Decisión dinámica (disponibilidad, costo, carga)
4. Override manual (usuario especifica LLM requerido)

## 🏗️ Arquitectura

### Layer 1: LLM Providers (Trait Pattern)

    #![allow(unused)]
    fn main() {
    pub enum LLMProvider {
        Claude {
            api_key: String,
            model: String,  // "opus-4", "sonnet-4", "haiku-3"
            max_tokens: usize,
        },
        OpenAI {
            api_key: String,
            model: String,  // "gpt-4", "gpt-4-turbo", "gpt-3.5-turbo"
            max_tokens: usize,
        },
        Gemini {
            api_key: String,
            model: String,  // "gemini-2.0-pro", "gemini-pro", "gemini-flash"
            max_tokens: usize,
        },
        Ollama {
            endpoint: String,  // "http://localhost:11434"
            model: String,     // "llama3.2", "mistral", "neural-chat"
            max_tokens: usize,
        },
    }

    pub trait LLMClient: Send + Sync {
        async fn complete(
            &self,
            prompt: String,
            context: Option<String>,
        ) -> anyhow::Result<String>;

        async fn stream(
            &self,
            prompt: String,
        ) -> anyhow::Result<tokio::sync::mpsc::Receiver<String>>;

        fn cost_per_1k_tokens(&self) -> f64;
        fn latency_ms(&self) -> u32;
        fn available(&self) -> bool;
    }
    }

### Layer 2: Task Context Classifier

    #![allow(unused)]
    fn main() {
    #[derive(Debug, Clone, PartialEq)]
    pub enum TaskType {
        // Code tasks
        CodeGeneration,
        CodeReview,
        CodeRefactor,
        UnitTest,
        Integration Test,

        // Analysis tasks
        ArchitectureDesign,
        SecurityAnalysis,
        PerformanceAnalysis,

        // Documentation
        DocumentGeneration,
        CodeDocumentation,
        APIDocumentation,

        // Search/RAG
        Embeddings,
        SemanticSearch,
        ContextRetrieval,

        // General
        GeneralQuery,
        Summarization,
        Translation,
    }

    #[derive(Debug, Clone)]
    pub struct TaskContext {
        pub task_type: TaskType,
        pub domain: String,              // "backend", "frontend", "infra"
        pub complexity: Complexity,       // Low, Medium, High, Critical
        pub quality_requirement: Quality, // Low, Medium, High, Critical
        pub latency_required_ms: u32,     // 500 = <500ms required
        pub budget_cents: Option<u32>,    // Cost limit in cents for 1k tokens
    }

    #[derive(Debug, Clone, PartialEq, PartialOrd)]
    pub enum Complexity {
        Low,
        Medium,
        High,
        Critical,
    }

    #[derive(Debug, Clone, PartialEq, PartialOrd)]
    pub enum Quality {
        Low,     // Quick & cheap
        Medium,  // Balanced
        High,    // Good quality
        Critical // Best possible
    }
    }

### Layer 3: Mapping Engine (Reglas Predefinidas)

    #![allow(unused)]
    fn main() {
    pub struct IAMapping {
        pub task_type: TaskType,
        pub primary: LLMProvider,
        pub fallback_order: Vec<LLMProvider>,
        pub reasoning: String,
        pub cost_estimate_per_task: f64,
    }

    pub static DEFAULT_MAPPINGS: &[IAMapping] = &[
        // Embeddings → Ollama (local, free)
        IAMapping {
            task_type: TaskType::Embeddings,
            primary: LLMProvider::Ollama {
                endpoint: "http://localhost:11434".to_string(),
                model: "nomic-embed-text".to_string(),
                max_tokens: 8192,
            },
            fallback_order: vec![
                LLMProvider::OpenAI {
                    api_key: "".to_string(),
                    model: "text-embedding-3-small".to_string(),
                    max_tokens: 8192,
                },
            ],
            reasoning: "Ollama local es gratis y rápido para embeddings. Fallback a OpenAI si Ollama no disponible".to_string(),
            cost_estimate_per_task: 0.0,  // Gratis localmente
        },

        // Code Generation → Claude Opus (máxima calidad)
        IAMapping {
            task_type: TaskType::CodeGeneration,
            primary: LLMProvider::Claude {
                api_key: "".to_string(),
                model: "opus-4".to_string(),
                max_tokens: 8000,
            },
            fallback_order: vec![
                LLMProvider::OpenAI {
                    api_key: "".to_string(),
                    model: "gpt-4".to_string(),
                    max_tokens: 8000,
                },
            ],
            reasoning: "Claude Opus mejor para código complejo. GPT-4 como fallback".to_string(),
            cost_estimate_per_task: 0.06,  // ~6 cents per 1k tokens
        },

        // Code Review → Claude Sonnet (balance calidad/costo)
        IAMapping {
            task_type: TaskType::CodeReview,
            primary: LLMProvider::Claude {
                api_key: "".to_string(),
                model: "sonnet-4".to_string(),
                max_tokens: 4000,
            },
            fallback_order: vec![
                LLMProvider::Gemini {
                    api_key: "".to_string(),
                    model: "gemini-pro".to_string(),
                    max_tokens: 4000,
                },
            ],
            reasoning: "Sonnet balance perfecto. Gemini como fallback".to_string(),
            cost_estimate_per_task: 0.015,
        },

        // Documentation → GPT-4 (mejor formato)
        IAMapping {
            task_type: TaskType::DocumentGeneration,
            primary: LLMProvider::OpenAI {
                api_key: "".to_string(),
                model: "gpt-4".to_string(),
                max_tokens: 4000,
            },
            fallback_order: vec![
                LLMProvider::Claude {
                    api_key: "".to_string(),
                    model: "sonnet-4".to_string(),
                    max_tokens: 4000,
                },
            ],
            reasoning: "GPT-4 mejor formato para docs. Claude como fallback".to_string(),
            cost_estimate_per_task: 0.03,
        },

        // Quick Queries → Gemini Flash (velocidad)
        IAMapping {
            task_type: TaskType::GeneralQuery,
            primary: LLMProvider::Gemini {
                api_key: "".to_string(),
                model: "gemini-flash-2.0".to_string(),
                max_tokens: 1000,
            },
            fallback_order: vec![
                LLMProvider::Ollama {
                    endpoint: "http://localhost:11434".to_string(),
                    model: "llama3.2".to_string(),
                    max_tokens: 1000,
                },
            ],
            reasoning: "Gemini Flash muy rápido. Ollama como fallback".to_string(),
            cost_estimate_per_task: 0.002,
        },
    ];
    }

### Layer 4: Routing Engine (Decisiones Dinámicas)

    #![allow(unused)]
    fn main() {
    pub struct LLMRouter {
        pub mappings: HashMap<TaskType, Vec<LLMProvider>>,
        pub providers: HashMap<String, Box<dyn LLMClient>>,
        pub cost_tracker: CostTracker,
        pub rate_limiter: RateLimiter,
    }

    impl LLMRouter {
        /// Routing decision: hybrid (rules + dynamic + override)
        pub async fn route(
            &mut self,
            context: TaskContext,
            override_llm: Option<LLMProvider>,
        ) -> anyhow::Result<LLMProvider> {
            // 1. Si hay override manual, usar ese
            if let Some(llm) = override_llm {
                self.cost_tracker.log_usage(&llm, &context);
                return Ok(llm);
            }

            // 2. Obtener mappings predefinidos
            let mut candidates = self.get_mapping(&context.task_type)?;

            // 3. Filtrar por disponibilidad (rate limits, latencia)
            candidates = self.filter_by_availability(candidates).await?;

            // 4. Filtrar por presupuesto si existe
            if let Some(budget) = context.budget_cents {
                candidates = candidates.into_iter()
                    .filter(|llm| llm.cost_per_1k_tokens() * 10.0 < budget as f64)
                    .collect();
            }

            // 5. Seleccionar por balance calidad/costo/latencia
            let selected = self.select_optimal(candidates, &context)?;

            self.cost_tracker.log_usage(&selected, &context);
            Ok(selected)
        }

        async fn filter_by_availability(
            &self,
            candidates: Vec<LLMProvider>,
        ) -> anyhow::Result<Vec<LLMProvider>> {
            let mut available = Vec::new();
            for llm in candidates {
                if self.rate_limiter.can_use(&llm).await? {
                    available.push(llm);
                }
            }
            Ok(available.is_empty() ? candidates : available)
        }

        fn select_optimal(
            &self,
            candidates: Vec<LLMProvider>,
            context: &TaskContext,
        ) -> anyhow::Result<LLMProvider> {
            // Scoring: quality * 0.4 + cost * 0.3 + latency * 0.3
            let best = candidates.iter().max_by(|a, b| {
                let score_a = self.score_llm(a, context);
                let score_b = self.score_llm(b, context);
                score_a.partial_cmp(&score_b).unwrap()
            });

            Ok(best.ok_or(anyhow::anyhow!("No LLM available"))?.clone())
        }

        fn score_llm(&self, llm: &LLMProvider, context: &TaskContext) -> f64 {
            let quality_score = match context.quality_requirement {
                Quality::Critical => 1.0,
                Quality::High => 0.9,
                Quality::Medium => 0.7,
                Quality::Low => 0.5,
            };

            let cost = llm.cost_per_1k_tokens();
            let cost_score = 1.0 / (1.0 + cost);  // Inverse: lower cost = higher score

            let latency = llm.latency_ms();
            let latency_score = 1.0 / (1.0 + latency as f64);

            quality_score * 0.4 + cost_score * 0.3 + latency_score * 0.3
        }
    }
    }

### Layer 5: Cost Tracking & Monitoring

    #![allow(unused)]
    fn main() {
    pub struct CostTracker {
        pub tasks_completed: HashMap<TaskType, u32>,
        pub total_tokens_used: u64,
        pub total_cost_cents: u32,
        pub cost_by_provider: HashMap<String, u32>,
        pub cost_by_task_type: HashMap<TaskType, u32>,
    }

    impl CostTracker {
        pub fn log_usage(&mut self, llm: &LLMProvider, context: &TaskContext) {
            let provider_name = llm.provider_name();
            let cost = (llm.cost_per_1k_tokens() * 10.0) as u32;  // Estimate per task

            *self.cost_by_provider.entry(provider_name).or_insert(0) += cost;
            *self.cost_by_task_type.entry(context.task_type.clone()).or_insert(0) += cost;
            self.total_cost_cents += cost;
            *self.tasks_completed.entry(context.task_type.clone()).or_insert(0) += 1;
        }

        pub fn monthly_cost_estimate(&self) -> f64 {
            self.total_cost_cents as f64 / 100.0  // Convert to dollars
        }

        pub fn generate_report(&self) -> String {
            format!(
                "Cost Report:\n  Total: ${:.2}\n  By Provider: {:?}\n  By Task: {:?}",
                self.monthly_cost_estimate(),
                self.cost_by_provider,
                self.cost_by_task_type
            )
        }
    }
    }

## 🔧 Routing: Tres Modos

### Modo 1: Reglas Estáticas (Default)

    #![allow(unused)]
    fn main() {
    // Automático, usa DEFAULT_MAPPINGS
    let router = LLMRouter::new();
    let llm = router.route(
        TaskContext {
            task_type: TaskType::CodeGeneration,
            domain: "backend".to_string(),
            complexity: Complexity::High,
            quality_requirement: Quality::High,
            latency_required_ms: 5000,
            budget_cents: None,
        },
        None,  // Sin override
    ).await?;
    // Resultado: Claude Opus (regla predefinida)
    }

### Modo 2: Decisión Dinámica (Smart)

    #![allow(unused)]
    fn main() {
    // Router evalúa disponibilidad, latencia, costo
    let router = LLMRouter::with_tracking();
    let llm = router.route(
        TaskContext {
            task_type: TaskType::CodeReview,
            domain: "frontend".to_string(),
            complexity: Complexity::Medium,
            quality_requirement: Quality::Medium,
            latency_required_ms: 2000,
            budget_cents: Some(20),  // Max 2 cents por task
        },
        None,
    ).await?;
    // Router elige entre Sonnet vs Gemini según disponibilidad y presupuesto
    }

### Modo 3: Override Manual (Control Total)

    #![allow(unused)]
    fn main() {
    // Usuario especifica exactamente qué LLM usar
    let llm = router.route(
        context,
        Some(LLMProvider::Claude {
            api_key: "sk-...".to_string(),
            model: "opus-4".to_string(),
            max_tokens: 8000,
        }),
    ).await?;
    // Usa exactamente lo especificado, registra en cost tracker
    }

## 📊 Configuración (vapora.toml)

    [llm_router]
    # Mapeos personalizados (override DEFAULT_MAPPINGS)
    [[llm_router.custom_mapping]]
    task_type = "CodeGeneration"
    primary_provider = "claude"
    primary_model = "opus-4"
    fallback_providers = ["openai:gpt-4"]

    # Proveedores disponibles
    [[llm_router.providers]]
    name = "claude"
    api_key = "${ANTHROPIC_API_KEY}"
    model_variants = ["opus-4", "sonnet-4", "haiku-3"]
    rate_limit = { tokens_per_minute = 1000000 }

    [[llm_router.providers]]
    name = "openai"
    api_key = "${OPENAI_API_KEY}"
    model_variants = ["gpt-4", "gpt-4-turbo"]
    rate_limit = { tokens_per_minute = 500000 }

    [[llm_router.providers]]
    name = "gemini"
    api_key = "${GEMINI_API_KEY}"
    model_variants = ["gemini-pro", "gemini-flash-2.0"]

    [[llm_router.providers]]
    name = "ollama"
    endpoint = "http://localhost:11434"
    model_variants = ["llama3.2", "mistral", "neural-chat"]
    rate_limit = { tokens_per_minute = 10000000 }  # Local, sin límites reales

    # Cost tracking
    [llm_router.cost_tracking]
    enabled = true
    warn_when_exceeds_cents = 1000  # Warn if daily cost > $10

## 🎯 Implementation Checklist

- Trait `LLMClient` + implementaciones (Claude, OpenAI, Gemini, Ollama)
- `TaskContext` y clasificación de tareas
- `IAMapping` y DEFAULT_MAPPINGS
- `LLMRouter` con routing híbrido
- Fallback automático + error handling
- `CostTracker` para monitoreo
- Config loading desde vapora.toml
- CLI: `vapora llm-router status` (ver providers, costos)
- Tests unitarios (routing logic)
- Integration tests (real providers)

## 📈 Success Metrics

✅ Routing decision < 100ms ✅ Fallback automático funciona ✅ Cost tracking preciso ✅ Documentación de costos por tarea ✅ Override manual siempre funciona ✅ Rate limiting respetado

Version: 0.1.0 Status: ✅ Specification Complete (VAPORA v1.0) Purpose: Multi-IA routing system para orquestación de agentes