provisioning/docs/src/ai/architecture.md

# AI Integration Architecture

## Overview

The provisioning platform's AI system provides intelligent capabilities for configuration generation, troubleshooting, and automation. The
architecture consists of multiple layers designed for reliability, security, and performance.

## Core Components - Production-Ready

### 1. AI Service (`provisioning/platform/ai-service`)

**Status**: ✅ Production-Ready (2,500+ lines Rust code)

The core AI service provides:
- Multi-provider LLM support (Anthropic Claude, OpenAI GPT-4, local models)
- Streaming response support for real-time feedback
- Request caching with LRU and semantic similarity
- Rate limiting and cost control
- Comprehensive error handling
- HTTP REST API on port 8083

**Supported Models**:
- Claude Sonnet 4, Claude Opus 4 (Anthropic)
- GPT-4 Turbo, GPT-4 (OpenAI)
- Llama 3, Mistral (local/on-premise)

### 2. RAG System (Retrieval-Augmented Generation)

**Status**: ✅ Production-Ready (22/22 tests passing)

The RAG system enables AI to access and reason over platform documentation:
- Vector embeddings via SurrealDB vector store
- Hybrid search: vector similarity + BM25 keyword search
- Document chunking (code and markdown aware)
- Relevance ranking and context selection
- Semantic caching for repeated queries

**Capabilities**:
```text
provisioning ai query "How do I set up Kubernetes?"
provisioning ai template "Describe my infrastructure"
```

### 3. MCP Server (Model Context Protocol)

**Status**: ✅ Production-Ready

Provides Model Context Protocol integration:
- Standardized tool interface for LLMs
- Complex workflow composition
- Integration with external AI systems (Claude, other LLMs)
- Tool calling for provisioning operations

### 4. CLI Integration

**Status**: ✅ Production-Ready

Interactive commands:
```text
provisioning ai template --prompt "Describe infrastructure"
provisioning ai query --prompt "Configuration question"
provisioning ai chat    # Interactive mode
```

**Configuration**:
```text
[ai]
enabled = true
provider = "anthropic"  # or "openai" or "local"
model = "claude-sonnet-4"

[ai.cache]
enabled = true
semantic_similarity = true
ttl_seconds = 3600

[ai.limits]
max_tokens = 4096
temperature = 0.7
```

## Planned Components - Q2 2025

### Autonomous Agents (typdialog-ag)

**Status**: 🔴 Planned

Self-directed agents for complex tasks:
- Multi-step workflow execution
- Decision making and adaptation
- Monitoring and self-healing recommendations

### AI-Assisted Forms (typdialog-ai)

**Status**: 🔴 Planned

Real-time AI suggestions in configuration forms:
- Context-aware field recommendations
- Validation error explanations
- Auto-completion for infrastructure patterns

### Advanced Features

- Fine-tuning capabilities for custom models
- Autonomous workflow execution with human approval
- Cedar authorization policies for AI actions
- Custom knowledge bases per workspace

## Architecture Diagram

```text
┌─────────────────────────────────────────────────┐
│  User Interface                                 │
│  ├── CLI (provisioning ai ...)                  │
│  ├── Web UI (typdialog)                         │
│  └── MCP Client (Claude, etc.)                  │
└──────────────┬──────────────────────────────────┘
               ↓
┌──────────────────────────────────────────────────┐
│  AI Service (Port 8083)                          │
│  ├── Request Router                              │
│  ├── Cache Layer (LRU + Semantic)                │
│  ├── Prompt Engineering                          │
│  └── Response Streaming                          │
└──────┬─────────────────┬─────────────────────────┘
       ↓                 ↓
┌─────────────┐  ┌──────────────────┐
│ RAG System  │  │ LLM Provider     │
│ SurrealDB   │  │ ├── Anthropic    │
│ Vector DB   │  │ ├── OpenAI       │
│ + BM25      │  │ └── Local Model  │
└─────────────┘  └──────────────────┘
       ↓                 ↓
┌──────────────────────────────────────┐
│  Cached Responses + Real Responses   │
│  Streamed to User                    │
└──────────────────────────────────────┘
```

## Performance Characteristics

|  | Metric | Value |  |
|  | -------- | ------- |  |
|  | Cold response (cache miss) | 2-5 seconds |  |
|  | Cached response | <500ms |  |
|  | Streaming start time | <1 second |  |
|  | AI service memory usage | ~200MB at rest |  |
|  | Cache size (configurable) | Up to 500MB |  |
|  | Vector DB (SurrealDB) | Included, auto-managed |  |

## Security Model

### Cedar Authorization

All AI operations controlled by Cedar policies:
- User role-based access control
- Operation-specific permissions
- Complete audit logging

### Secret Protection

- Secrets never sent to external LLMs
- PII/sensitive data sanitized before API calls
- Encryption at rest in local cache
- HSM support for key storage

### Local Model Support

Air-gapped deployments:
- On-premise LLM models (Llama 3, Mistral)
- Zero external API calls
- Full data privacy compliance
- Ideal for classified environments

## Configuration

See [Configuration Guide](configuration.md) for:
- LLM provider setup
- Cache configuration
- Cost limits and budgets
- Security policies

## Related Documentation

- [RAG System](rag-system.md) - Retrieval implementation details
- [Security Policies](security-policies.md) - Authorization and safety controls
- [Configuration Guide](configuration.md) - Setup instructions
- [ADR-015](../architecture/adr/adr-015-ai-integration-architecture.md) - Design decisions

---

**Last Updated**: 2025-01-13
**Status**: ✅ Production-Ready (core system)
**Test Coverage**: 22/22 tests passing