# AI Integration Architecture\n\n## Overview\n\nThe provisioning platform's AI system provides intelligent capabilities for configuration generation, troubleshooting, and automation. The\narchitecture consists of multiple layers designed for reliability, security, and performance.\n\n## Core Components - Production-Ready\n\n### 1. AI Service (`provisioning/platform/ai-service`)\n\n**Status**: ✅ Production-Ready (2,500+ lines Rust code)\n\nThe core AI service provides:\n- Multi-provider LLM support (Anthropic Claude, OpenAI GPT-4, local models)\n- Streaming response support for real-time feedback\n- Request caching with LRU and semantic similarity\n- Rate limiting and cost control\n- Comprehensive error handling\n- HTTP REST API on port 8083\n\n**Supported Models**:\n- Claude Sonnet 4, Claude Opus 4 (Anthropic)\n- GPT-4 Turbo, GPT-4 (OpenAI)\n- Llama 3, Mistral (local/on-premise)\n\n### 2. RAG System (Retrieval-Augmented Generation)\n\n**Status**: ✅ Production-Ready (22/22 tests passing)\n\nThe RAG system enables AI to access and reason over platform documentation:\n- Vector embeddings via SurrealDB vector store\n- Hybrid search: vector similarity + BM25 keyword search\n- Document chunking (code and markdown aware)\n- Relevance ranking and context selection\n- Semantic caching for repeated queries\n\n**Capabilities**:\n```\nprovisioning ai query "How do I set up Kubernetes?"\nprovisioning ai template "Describe my infrastructure"\n```\n\n### 3. MCP Server (Model Context Protocol)\n\n**Status**: ✅ Production-Ready\n\nProvides Model Context Protocol integration:\n- Standardized tool interface for LLMs\n- Complex workflow composition\n- Integration with external AI systems (Claude, other LLMs)\n- Tool calling for provisioning operations\n\n### 4. CLI Integration\n\n**Status**: ✅ Production-Ready\n\nInteractive commands:\n```\nprovisioning ai template --prompt "Describe infrastructure"\nprovisioning ai query --prompt "Configuration question"\nprovisioning ai chat    # Interactive mode\n```\n\n**Configuration**:\n```\n[ai]\nenabled = true\nprovider = "anthropic"  # or "openai" or "local"\nmodel = "claude-sonnet-4"\n\n[ai.cache]\nenabled = true\nsemantic_similarity = true\nttl_seconds = 3600\n\n[ai.limits]\nmax_tokens = 4096\ntemperature = 0.7\n```\n\n## Planned Components - Q2 2025\n\n### Autonomous Agents (typdialog-ag)\n\n**Status**: 🔴 Planned\n\nSelf-directed agents for complex tasks:\n- Multi-step workflow execution\n- Decision making and adaptation\n- Monitoring and self-healing recommendations\n\n### AI-Assisted Forms (typdialog-ai)\n\n**Status**: 🔴 Planned\n\nReal-time AI suggestions in configuration forms:\n- Context-aware field recommendations\n- Validation error explanations\n- Auto-completion for infrastructure patterns\n\n### Advanced Features\n\n- Fine-tuning capabilities for custom models\n- Autonomous workflow execution with human approval\n- Cedar authorization policies for AI actions\n- Custom knowledge bases per workspace\n\n## Architecture Diagram\n\n```\n┌─────────────────────────────────────────────────┐\n│  User Interface                                 │\n│  ├── CLI (provisioning ai ...)                  │\n│  ├── Web UI (typdialog)                         │\n│  └── MCP Client (Claude, etc.)                  │\n└──────────────┬──────────────────────────────────┘\n               ↓\n┌──────────────────────────────────────────────────┐\n│  AI Service (Port 8083)                          │\n│  ├── Request Router                             │\n│  ├── Cache Layer (LRU + Semantic)              │\n│  ├── Prompt Engineering                         │\n│  └── Response Streaming                         │\n└──────┬─────────────────┬─────────────────────────┘\n       ↓                 ↓\n┌─────────────┐  ┌──────────────────┐\n│ RAG System  │  │ LLM Provider     │\n│ SurrealDB   │  │ ├── Anthropic    │\n│ Vector DB   │  │ ├── OpenAI       │\n│ + BM25      │  │ └── Local Model  │\n└─────────────┘  └──────────────────┘\n       ↓                 ↓\n┌──────────────────────────────────────┐\n│  Cached Responses + Real Responses   │\n│  Streamed to User                    │\n└──────────────────────────────────────┘\n```\n\n## Performance Characteristics\n\n|  | Metric | Value |  |\n|  | -------- | ------- |  |\n|  | Cold response (cache miss) | 2-5 seconds |  |\n|  | Cached response | <500ms |  |\n|  | Streaming start time | <1 second |  |\n|  | AI service memory usage | ~200MB at rest |  |\n|  | Cache size (configurable) | Up to 500MB |  |\n|  | Vector DB (SurrealDB) | Included, auto-managed |  |\n\n## Security Model\n\n### Cedar Authorization\n\nAll AI operations controlled by Cedar policies:\n- User role-based access control\n- Operation-specific permissions\n- Complete audit logging\n\n### Secret Protection\n\n- Secrets never sent to external LLMs\n- PII/sensitive data sanitized before API calls\n- Encryption at rest in local cache\n- HSM support for key storage\n\n### Local Model Support\n\nAir-gapped deployments:\n- On-premise LLM models (Llama 3, Mistral)\n- Zero external API calls\n- Full data privacy compliance\n- Ideal for classified environments\n\n## Configuration\n\nSee [Configuration Guide](configuration.md) for:\n- LLM provider setup\n- Cache configuration\n- Cost limits and budgets\n- Security policies\n\n## Related Documentation\n\n- [RAG System](rag-system.md) - Retrieval implementation details\n- [Security Policies](security-policies.md) - Authorization and safety controls\n- [Configuration Guide](configuration.md) - Setup instructions\n- [ADR-015](../architecture/adr/adr-015-ai-integration-architecture.md) - Design decisions\n\n---\n\n**Last Updated**: 2025-01-13\n**Status**: ✅ Production-Ready (core system)\n**Test Coverage**: 22/22 tests passing