## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# ADR-015: Three-Tier Budget Enforcement con Auto-Fallback

Status: Accepted | Implemented Date: 2024-11-01 Deciders: Cost Architecture Team Technical Story: Preventing LLM spend overruns with dual time windows and graceful degradation

## Decision

Implementar three-tier budget enforcement con dual time windows (monthly + weekly) y automatic fallback a Ollama.

## Rationale

1. Dual Windows: Previene tanto overspend a largo plazo (monthly) como picos (weekly)
2. Three States: Normal → Near-threshold → Exceeded (progressive restriction)
3. Auto-Fallback: Usar Ollama ($0) cuando budget exceeded (graceful degradation)
4. Per-Role Limits: Budget distinto por rol (arquitecto vs developer vs reviewer)

## Alternatives Considered

### ❌ Monthly Only

- Pros: Simple
- Cons: Allow weekly spikes, late-month overspend

### ❌ Weekly Only

- Pros: Catches spikes
- Cons: No protection for slow bleed, fragmented budget

### ✅ Dual Windows + Auto-Fallback (CHOSEN)

- Protege contra ambos spikes y long-term overspend

## Trade-offs

Pros:

- ✅ Protection against both spike and gradual overspend
- ✅ Progressive alerts (normal → near → exceeded)
- ✅ Automatic fallback prevents hard stops
- ✅ Per-role customization
- ✅ Quality degrades gracefully

Cons:

- ⚠️ Alert fatigue possible if thresholds set too tight
- ⚠️ Fallback to Ollama may reduce quality
- ⚠️ Configuration complexity (two threshold sets)

## Implementation

Budget Configuration:

    # config/budget.toml

    [[role_budgets]]
    role = "architect"
    monthly_budget_usd = 1000
    weekly_budget_usd = 250

    [[role_budgets]]
    role = "developer"
    monthly_budget_usd = 500
    weekly_budget_usd = 125

    [[role_budgets]]
    role = "reviewer"
    monthly_budget_usd = 200
    weekly_budget_usd = 50

    # Enforcement thresholds
    [enforcement]
    normal_threshold = 0.80       # < 80%: Use optimal provider
    near_threshold = 1.0          # 80-100%: Cheaper providers
    exceeded_threshold = 1.0      # > 100%: Fallback to Ollama

    [alerts]
    near_threshold_alert = true
    exceeded_alert = true
    alert_channels = ["slack", "email"]

Budget Tracking Model:

    #![allow(unused)]
    fn main() {
    // crates/vapora-llm-router/src/budget.rs
    pub struct BudgetState {
        pub role: String,
        pub monthly_spent_cents: u32,
        pub monthly_budget_cents: u32,
        pub weekly_spent_cents: u32,
        pub weekly_budget_cents: u32,
        pub last_reset_week: Week,
    }

    #[derive(Debug, Clone, Copy, PartialEq, Eq)]
    pub enum EnforcementState {
        Normal,           // < 80%: Use optimal provider
        NearThreshold,    // 80-100%: Prefer cheaper
        Exceeded,         // > 100%: Fallback to Ollama
    }

    impl BudgetState {
        pub fn monthly_percentage(&self) -> f32 {
            (self.monthly_spent_cents as f32) / (self.monthly_budget_cents as f32)
        }

        pub fn weekly_percentage(&self) -> f32 {
            (self.weekly_spent_cents as f32) / (self.weekly_budget_cents as f32)
        }

        pub fn enforcement_state(&self) -> EnforcementState {
            let monthly_pct = self.monthly_percentage();
            let weekly_pct = self.weekly_percentage();

            // Use more restrictive of two
            let most_restrictive = monthly_pct.max(weekly_pct);

            if most_restrictive < 0.80 {
                EnforcementState::Normal
            } else if most_restrictive < 1.0 {
                EnforcementState::NearThreshold
            } else {
                EnforcementState::Exceeded
            }
        }
    }
    }

Budget Enforcement in Router:

    #![allow(unused)]
    fn main() {
    pub async fn route_with_budget(
        task: &Task,
        user_role: &str,
        budget_state: &mut BudgetState,
    ) -> Result<String> {
        // Check budget state
        let enforcement = budget_state.enforcement_state();

        match enforcement {
            EnforcementState::Normal => {
                // Use optimal provider (Claude, GPT-4)
                let provider = select_optimal_provider(task).await?;
                execute_with_provider(task, &provider, budget_state).await
            }
            EnforcementState::NearThreshold => {
                // Alert user, prefer cheaper providers
                alert_near_threshold(user_role, budget_state)?;
                let provider = select_cheap_provider(task).await?;
                execute_with_provider(task, &provider, budget_state).await
            }
            EnforcementState::Exceeded => {
                // Alert, fallback to Ollama
                alert_exceeded(user_role, budget_state)?;
                let provider = "ollama"; // Free
                execute_with_provider(task, provider, budget_state).await
            }
        }
    }

    async fn execute_with_provider(
        task: &Task,
        provider: &str,
        budget_state: &mut BudgetState,
    ) -> Result<String> {
        let response = call_provider(task, provider).await?;
        let cost_cents = estimate_cost(&response, provider)?;

        // Update budget
        budget_state.monthly_spent_cents += cost_cents;
        budget_state.weekly_spent_cents += cost_cents;

        // Log for audit
        log_budget_usage(task.id, provider, cost_cents)?;

        Ok(response)
    }
    }

Reset Logic:

    #![allow(unused)]
    fn main() {
    pub async fn reset_budget_weekly(db: &Surreal<Ws>) -> Result<()> {
        let now = Utc::now();
        let current_week = week_number(now);

        let budgets = db.query(
            "SELECT * FROM role_budgets WHERE last_reset_week < $1"
        )
        .bind(current_week)
        .await?;

        for mut budget in budgets {
            budget.weekly_spent_cents = 0;
            budget.last_reset_week = current_week;
            db.update(&budget.id).content(&budget).await?;
        }

        Ok(())
    }
    }

Key Files:

- `/crates/vapora-llm-router/src/budget.rs` (budget tracking)
- `/crates/vapora-llm-router/src/cost_tracker.rs` (cost calculation)
- `/crates/vapora-llm-router/src/router.rs` (enforcement logic)
- `/config/budget.toml` (configuration)

## Verification

    # Test budget percentage calculation
    cargo test -p vapora-llm-router test_budget_percentage

    # Test enforcement states
    cargo test -p vapora-llm-router test_enforcement_states

    # Test normal → near-threshold transition
    cargo test -p vapora-llm-router test_near_threshold_alert

    # Test exceeded → fallback to Ollama
    cargo test -p vapora-llm-router test_budget_exceeded_fallback

    # Test weekly reset
    cargo test -p vapora-llm-router test_weekly_budget_reset

    # Integration: full budget lifecycle
    cargo test -p vapora-llm-router test_budget_full_cycle

Expected Output:

- Budget percentages calculated correctly
- Enforcement state transitions as budget fills
- Near-threshold alerts triggered at 80%
- Fallback to Ollama when exceeded 100%
- Weekly reset clears weekly budget
- Monthly budget accumulates across weeks
- All transitions logged for audit

## Consequences

### Financial

- Predictable monthly costs (bounded by monthly_budget)
- Alert on near-threshold prevents surprises
- Auto-fallback protects against runaway spend

### User Experience

- Quality degrades gracefully (not hard stop)
- Users can continue working (Ollama fallback)
- Alerts notify of budget status

### Operations

- Budget resets automated (weekly)
- Per-role customization allows differentiation
- Cost reports broken down by role

### Monitoring

- Track which roles consuming most budget
- Identify unusual spend patterns
- Forecast end-of-month spend

## References

- `/crates/vapora-llm-router/src/budget.rs` (budget implementation)
- `/crates/vapora-llm-router/src/cost_tracker.rs` (cost tracking)
- `/config/budget.toml` (configuration)
- ADR-007 (Multi-Provider LLM)
- ADR-016 (Cost Efficiency Ranking)

Related ADRs: ADR-007 (Multi-Provider), ADR-016 (Cost Efficiency), ADR-012 (Routing Tiers)