provisioning/docs/src/ai/architecture.md

AI Integration Architecture

Overview

The provisioning platform's AI system provides intelligent capabilities for configuration generation, troubleshooting, and automation. The architecture consists of multiple layers designed for reliability, security, and performance.

Core Components - Production-Ready

1. AI Service (provisioning/platform/ai-service)

Status: ✅ Production-Ready (2,500+ lines Rust code)

The core AI service provides:

• Multi-provider LLM support (Anthropic Claude, OpenAI GPT-4, local models)
• Streaming response support for real-time feedback
• Request caching with LRU and semantic similarity
• Rate limiting and cost control
• Comprehensive error handling
• HTTP REST API on port 8083

Supported Models:

• Claude Sonnet 4, Claude Opus 4 (Anthropic)
• GPT-4 Turbo, GPT-4 (OpenAI)
• Llama 3, Mistral (local/on-premise)

2. RAG System (Retrieval-Augmented Generation)

Status: ✅ Production-Ready (22/22 tests passing)

The RAG system enables AI to access and reason over platform documentation:

• Vector embeddings via SurrealDB vector store
• Hybrid search: vector similarity + BM25 keyword search
• Document chunking (code and markdown aware)
• Relevance ranking and context selection
• Semantic caching for repeated queries

Capabilities:

provisioning ai query "How do I set up Kubernetes?"
provisioning ai template "Describe my infrastructure"

3. MCP Server (Model Context Protocol)

Status: ✅ Production-Ready

Provides Model Context Protocol integration:

• Standardized tool interface for LLMs
• Complex workflow composition
• Integration with external AI systems (Claude, other LLMs)
• Tool calling for provisioning operations

4. CLI Integration

Status: ✅ Production-Ready

Interactive commands:

provisioning ai template --prompt "Describe infrastructure"
provisioning ai query --prompt "Configuration question"
provisioning ai chat    # Interactive mode

Configuration:

[ai]
enabled = true
provider = "anthropic"  # or "openai" or "local"
model = "claude-sonnet-4"

[ai.cache]
enabled = true
semantic_similarity = true
ttl_seconds = 3600

[ai.limits]
max_tokens = 4096
temperature = 0.7

Planned Components - Q2 2025

Autonomous Agents (typdialog-ag)

Status: 🔴 Planned

Self-directed agents for complex tasks:

• Multi-step workflow execution
• Decision making and adaptation
• Monitoring and self-healing recommendations

AI-Assisted Forms (typdialog-ai)

Status: 🔴 Planned

Real-time AI suggestions in configuration forms:

• Context-aware field recommendations
• Validation error explanations
• Auto-completion for infrastructure patterns

Advanced Features

• Fine-tuning capabilities for custom models
• Autonomous workflow execution with human approval
• Cedar authorization policies for AI actions
• Custom knowledge bases per workspace

Architecture Diagram

┌─────────────────────────────────────────────────┐
│  User Interface                                 │
│  ├── CLI (provisioning ai ...)                  │
│  ├── Web UI (typdialog)                         │
│  └── MCP Client (Claude, etc.)                  │
└──────────────┬──────────────────────────────────┘
               ↓
┌──────────────────────────────────────────────────┐
│  AI Service (Port 8083)                          │
│  ├── Request Router                              │
│  ├── Cache Layer (LRU + Semantic)                │
│  ├── Prompt Engineering                          │
│  └── Response Streaming                          │
└──────┬─────────────────┬─────────────────────────┘
       ↓                 ↓
┌─────────────┐  ┌──────────────────┐
│ RAG System  │  │ LLM Provider     │
│ SurrealDB   │  │ ├── Anthropic    │
│ Vector DB   │  │ ├── OpenAI       │
│ + BM25      │  │ └── Local Model  │
└─────────────┘  └──────────────────┘
       ↓                 ↓
┌──────────────────────────────────────┐
│  Cached Responses + Real Responses   │
│  Streamed to User                    │
└──────────────────────────────────────┘

Performance Characteristics

| | Metric | Value | | | | -------- | ------- | | | | Cold response (cache miss) | 2-5 seconds | | | | Cached response | <500ms | | | | Streaming start time | <1 second | | | | AI service memory usage | ~200MB at rest | | | | Cache size (configurable) | Up to 500MB | | | | Vector DB (SurrealDB) | Included, auto-managed | |

Security Model

Cedar Authorization

All AI operations controlled by Cedar policies:

• User role-based access control
• Operation-specific permissions
• Complete audit logging

Secret Protection

• Secrets never sent to external LLMs
• PII/sensitive data sanitized before API calls
• Encryption at rest in local cache
• HSM support for key storage

Local Model Support

Air-gapped deployments:

• On-premise LLM models (Llama 3, Mistral)
• Zero external API calls
• Full data privacy compliance
• Ideal for classified environments

Configuration

See Configuration Guide for:

• LLM provider setup
• Cache configuration
• Cost limits and budgets
• Security policies

Related Documentation

• RAG System - Retrieval implementation details
• Security Policies - Authorization and safety controls
• Configuration Guide - Setup instructions
• ADR-015 - Design decisions

---

Last Updated: 2025-01-13 Status: ✅ Production-Ready (core system) Test Coverage: 22/22 tests passing