1150 lines
26 KiB
Markdown
1150 lines
26 KiB
Markdown
# Complete Deployment Guide: From Scratch to Production
|
||
|
||
**Version**: 3.5.0
|
||
**Last Updated**: 2025-10-09
|
||
**Estimated Time**: 30-60 minutes
|
||
**Difficulty**: Beginner to Intermediate
|
||
|
||
---
|
||
|
||
## Table of Contents
|
||
|
||
1. [Prerequisites](#prerequisites)
|
||
2. [Step 1: Install Nushell](#step-1-install-nushell)
|
||
3. [Step 2: Install Nushell Plugins (Recommended)](#step-2-install-nushell-plugins-recommended)
|
||
4. [Step 3: Install Required Tools](#step-3-install-required-tools)
|
||
5. [Step 4: Clone and Setup Project](#step-4-clone-and-setup-project)
|
||
6. [Step 5: Initialize Workspace](#step-5-initialize-workspace)
|
||
7. [Step 6: Configure Environment](#step-6-configure-environment)
|
||
8. [Step 7: Discover and Load Modules](#step-7-discover-and-load-modules)
|
||
9. [Step 8: Validate Configuration](#step-8-validate-configuration)
|
||
10. [Step 9: Deploy Servers](#step-9-deploy-servers)
|
||
11. [Step 10: Install Task Services](#step-10-install-task-services)
|
||
12. [Step 11: Create Clusters](#step-11-create-clusters)
|
||
13. [Step 12: Verify Deployment](#step-12-verify-deployment)
|
||
14. [Step 13: Post-Deployment](#step-13-post-deployment)
|
||
15. [Troubleshooting](#troubleshooting)
|
||
16. [Next Steps](#next-steps)
|
||
|
||
---
|
||
|
||
## Prerequisites
|
||
|
||
Before starting, ensure you have:
|
||
|
||
- ✅ **Operating System**: macOS, Linux, or Windows (WSL2 recommended)
|
||
- ✅ **Administrator Access**: Ability to install software and configure system
|
||
- ✅ **Internet Connection**: For downloading dependencies and accessing cloud providers
|
||
- ✅ **Cloud Provider Credentials**: UpCloud, Hetzner, AWS, or local development environment
|
||
- ✅ **Basic Terminal Knowledge**: Comfortable running shell commands
|
||
- ✅ **Text Editor**: vim, nano, Zed, VSCode, or your preferred editor
|
||
|
||
### Recommended Hardware
|
||
|
||
- **CPU**: 2+ cores
|
||
- **RAM**: 8 GB minimum, 16 GB recommended
|
||
- **Disk**: 20 GB free space minimum
|
||
|
||
---
|
||
|
||
## Step 1: Install Nushell
|
||
|
||
Nushell 0.109.1+ is the primary shell and scripting language for the provisioning platform.
|
||
|
||
### macOS (via Homebrew)
|
||
|
||
```bash
|
||
# Install Nushell
|
||
brew install nushell
|
||
|
||
# Verify installation
|
||
nu --version
|
||
# Expected: 0.109.1 or higher
|
||
```
|
||
|
||
### Linux (via Package Manager)
|
||
|
||
**Ubuntu/Debian:**
|
||
|
||
```bash
|
||
# Add Nushell repository
|
||
curl -fsSL https://starship.rs/install.sh | bash
|
||
|
||
# Install Nushell
|
||
sudo apt update
|
||
sudo apt install nushell
|
||
|
||
# Verify installation
|
||
nu --version
|
||
```
|
||
|
||
**Fedora:**
|
||
|
||
```bash
|
||
sudo dnf install nushell
|
||
nu --version
|
||
```
|
||
|
||
**Arch Linux:**
|
||
|
||
```bash
|
||
sudo pacman -S nushell
|
||
nu --version
|
||
```
|
||
|
||
### Linux/macOS (via Cargo)
|
||
|
||
```bash
|
||
# Install Rust (if not already installed)
|
||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
||
source $HOME/.cargo/env
|
||
|
||
# Install Nushell
|
||
cargo install nu --locked
|
||
|
||
# Verify installation
|
||
nu --version
|
||
```
|
||
|
||
### Windows (via Winget)
|
||
|
||
```powershell
|
||
# Install Nushell
|
||
winget install nushell
|
||
|
||
# Verify installation
|
||
nu --version
|
||
```
|
||
|
||
### Configure Nushell
|
||
|
||
```bash
|
||
# Start Nushell
|
||
nu
|
||
|
||
# Configure (creates default config if not exists)
|
||
config nu
|
||
```
|
||
|
||
---
|
||
|
||
## Step 2: Install Nushell Plugins (Recommended)
|
||
|
||
Native plugins provide **10-50x performance improvement** for authentication, KMS, and orchestrator operations.
|
||
|
||
### Why Install Plugins
|
||
|
||
**Performance Gains:**
|
||
|
||
- 🚀 **KMS operations**: ~5 ms vs ~50 ms (10x faster)
|
||
- 🚀 **Orchestrator queries**: ~1 ms vs ~30 ms (30x faster)
|
||
- 🚀 **Batch encryption**: 100 files in 0.5s vs 5s (10x faster)
|
||
|
||
**Benefits:**
|
||
|
||
- ✅ Native Nushell integration (pipelines, data structures)
|
||
- ✅ OS keyring for secure token storage
|
||
- ✅ Offline capability (Age encryption, local orchestrator)
|
||
- ✅ Graceful fallback to HTTP if not installed
|
||
|
||
### Prerequisites for Building Plugins
|
||
|
||
```bash
|
||
# Install Rust toolchain (if not already installed)
|
||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
||
source $HOME/.cargo/env
|
||
rustc --version
|
||
# Expected: rustc 1.75+ or higher
|
||
|
||
# Linux only: Install development packages
|
||
sudo apt install libssl-dev pkg-config # Ubuntu/Debian
|
||
sudo dnf install openssl-devel # Fedora
|
||
|
||
# Linux only: Install keyring service (required for auth plugin)
|
||
sudo apt install gnome-keyring # Ubuntu/Debian (GNOME)
|
||
sudo apt install kwalletmanager # Ubuntu/Debian (KDE)
|
||
```
|
||
|
||
### Build Plugins
|
||
|
||
```bash
|
||
# Navigate to plugins directory
|
||
cd provisioning/core/plugins/nushell-plugins
|
||
|
||
# Build all three plugins in release mode (optimized)
|
||
cargo build --release --all
|
||
|
||
# Expected output:
|
||
# Compiling nu_plugin_auth v0.1.0
|
||
# Compiling nu_plugin_kms v0.1.0
|
||
# Compiling nu_plugin_orchestrator v0.1.0
|
||
# Finished release [optimized] target(s) in 2m 15s
|
||
```
|
||
|
||
**Build time**: ~2-5 minutes depending on hardware
|
||
|
||
### Register Plugins with Nushell
|
||
|
||
```bash
|
||
# Register all three plugins (full paths recommended)
|
||
plugin add $PWD/target/release/nu_plugin_auth
|
||
plugin add $PWD/target/release/nu_plugin_kms
|
||
plugin add $PWD/target/release/nu_plugin_orchestrator
|
||
|
||
# Alternative (from plugins directory)
|
||
plugin add target/release/nu_plugin_auth
|
||
plugin add target/release/nu_plugin_kms
|
||
plugin add target/release/nu_plugin_orchestrator
|
||
```
|
||
|
||
### Verify Plugin Installation
|
||
|
||
```bash
|
||
# List registered plugins
|
||
plugin list | where name =~ "auth|kms|orch"
|
||
|
||
# Expected output:
|
||
# ╭───┬─────────────────────────┬─────────┬───────────────────────────────────╮
|
||
# │ # │ name │ version │ filename │
|
||
# ├───┼─────────────────────────┼─────────┼───────────────────────────────────┤
|
||
# │ 0 │ nu_plugin_auth │ 0.1.0 │ .../nu_plugin_auth │
|
||
# │ 1 │ nu_plugin_kms │ 0.1.0 │ .../nu_plugin_kms │
|
||
# │ 2 │ nu_plugin_orchestrator │ 0.1.0 │ .../nu_plugin_orchestrator │
|
||
# ╰───┴─────────────────────────┴─────────┴───────────────────────────────────╯
|
||
|
||
# Test each plugin
|
||
auth --help # Should show auth commands
|
||
kms --help # Should show kms commands
|
||
orch --help # Should show orch commands
|
||
```
|
||
|
||
### Configure Plugin Environments
|
||
|
||
```bash
|
||
# Add to ~/.config/nushell/env.nu
|
||
$env.CONTROL_CENTER_URL = "http://localhost:3000"
|
||
$env.RUSTYVAULT_ADDR = "http://localhost:8200"
|
||
$env.RUSTYVAULT_TOKEN = "your-vault-token-here"
|
||
$env.ORCHESTRATOR_DATA_DIR = "provisioning/platform/orchestrator/data"
|
||
|
||
# For Age encryption (local development)
|
||
$env.AGE_IDENTITY = $"($env.HOME)/.age/key.txt"
|
||
$env.AGE_RECIPIENT = "age1xxxxxxxxx" # Replace with your public key
|
||
```
|
||
|
||
### Test Plugins (Quick Smoke Test)
|
||
|
||
```bash
|
||
# Test KMS plugin (requires backend configured)
|
||
kms status
|
||
# Expected: { backend: "rustyvault", status: "healthy", ... }
|
||
# Or: Error if backend not configured (OK for now)
|
||
|
||
# Test orchestrator plugin (reads local files)
|
||
orch status
|
||
# Expected: { active_tasks: 0, completed_tasks: 0, health: "healthy" }
|
||
# Or: Error if orchestrator not started yet (OK for now)
|
||
|
||
# Test auth plugin (requires control center)
|
||
auth verify
|
||
# Expected: { active: false }
|
||
# Or: Error if control center not running (OK for now)
|
||
```
|
||
|
||
**Note**: It's OK if plugins show errors at this stage. We'll configure backends and services later.
|
||
|
||
### Skip Plugins (Not Recommended)
|
||
|
||
If you want to skip plugin installation for now:
|
||
|
||
- ✅ All features work via HTTP API (slower but functional)
|
||
- ⚠️ You'll miss 10-50x performance improvements
|
||
- ⚠️ No offline capability for KMS/orchestrator
|
||
- ℹ️ You can install plugins later anytime
|
||
|
||
To use HTTP fallback:
|
||
|
||
```bash
|
||
# System automatically uses HTTP if plugins not available
|
||
# No configuration changes needed
|
||
```
|
||
|
||
---
|
||
|
||
## Step 3: Install Required Tools
|
||
|
||
### Essential Tools
|
||
|
||
**SOPS (Secrets Management)**
|
||
|
||
```bash
|
||
# macOS
|
||
brew install sops
|
||
|
||
# Linux
|
||
wget https://github.com/mozilla/sops/releases/download/v3.10.2/sops-v3.10.2.linux.amd64
|
||
sudo mv sops-v3.10.2.linux.amd64 /usr/local/bin/sops
|
||
sudo chmod +x /usr/local/bin/sops
|
||
|
||
# Verify
|
||
sops --version
|
||
# Expected: 3.10.2 or higher
|
||
```
|
||
|
||
**Age (Encryption Tool)**
|
||
|
||
```bash
|
||
# macOS
|
||
brew install age
|
||
|
||
# Linux
|
||
sudo apt install age # Ubuntu/Debian
|
||
sudo dnf install age # Fedora
|
||
|
||
# Or from source
|
||
go install filippo.io/age/cmd/...@latest
|
||
|
||
# Verify
|
||
age --version
|
||
# Expected: 1.2.1 or higher
|
||
|
||
# Generate Age key (for local encryption)
|
||
age-keygen -o ~/.age/key.txt
|
||
cat ~/.age/key.txt
|
||
# Save the public key (age1...) for later
|
||
```
|
||
|
||
### Optional but Recommended Tools
|
||
|
||
**K9s (Kubernetes Management)**
|
||
|
||
```bash
|
||
# macOS
|
||
brew install k9s
|
||
|
||
# Linux
|
||
curl -sS https://webinstall.dev/k9s | bash
|
||
|
||
# Verify
|
||
k9s version
|
||
# Expected: 0.50.6 or higher
|
||
```
|
||
|
||
**glow (Markdown Renderer)**
|
||
|
||
```bash
|
||
# macOS
|
||
brew install glow
|
||
|
||
# Linux
|
||
sudo apt install glow # Ubuntu/Debian
|
||
sudo dnf install glow # Fedora
|
||
|
||
# Verify
|
||
glow --version
|
||
```
|
||
|
||
---
|
||
|
||
## Step 4: Clone and Setup Project
|
||
|
||
### Clone Repository
|
||
|
||
```bash
|
||
# Clone project
|
||
git clone https://github.com/your-org/project-provisioning.git
|
||
cd project-provisioning
|
||
|
||
# Or if already cloned, update to latest
|
||
git pull origin main
|
||
```
|
||
|
||
### Add CLI to PATH (Optional)
|
||
|
||
```bash
|
||
# Add to ~/.bashrc or ~/.zshrc
|
||
export PATH="$PATH:/Users/Akasha/project-provisioning/provisioning/core/cli"
|
||
|
||
# Or create symlink
|
||
sudo ln -s /Users/Akasha/project-provisioning/provisioning/core/cli/provisioning /usr/local/bin/provisioning
|
||
|
||
# Verify
|
||
provisioning version
|
||
# Expected: 3.5.0
|
||
```
|
||
|
||
---
|
||
|
||
## Step 5: Initialize Workspace
|
||
|
||
A workspace is a self-contained environment for managing infrastructure.
|
||
|
||
### Create New Workspace
|
||
|
||
```bash
|
||
# Initialize new workspace
|
||
provisioning workspace init --name production
|
||
|
||
# Or use interactive mode
|
||
provisioning workspace init
|
||
# Name: production
|
||
# Description: Production infrastructure
|
||
# Provider: upcloud
|
||
```
|
||
|
||
**What this creates:**
|
||
|
||
The new workspace initialization now generates **Nickel configuration files** for type-safe, schema-validated infrastructure definitions:
|
||
|
||
```plaintext
|
||
workspace/
|
||
├── config/
|
||
│ ├── config.ncl # Master Nickel configuration (type-safe)
|
||
│ ├── providers/
|
||
│ │ └── upcloud.toml # Provider-specific settings
|
||
│ ├── platform/ # Platform service configs
|
||
│ └── kms.toml # Key management settings
|
||
├── infra/
|
||
│ └── default/
|
||
│ ├── main.ncl # Infrastructure entry point
|
||
│ └── servers.ncl # Server definitions
|
||
├── docs/ # Auto-generated guides
|
||
└── workspace.nu # Workspace utility scripts
|
||
```
|
||
|
||
### Workspace Configuration Format
|
||
|
||
The workspace configuration uses **Nickel (type-safe, validated)**. This provides:
|
||
|
||
- ✅ **Type Safety**: Schema validation catches errors at load time
|
||
- ✅ **Lazy Evaluation**: Only computes what's needed
|
||
- ✅ **Validation**: Record merging, required fields, constraints
|
||
- ✅ **Documentation**: Self-documenting with records
|
||
|
||
**Example Nickel config** (`config.ncl`):
|
||
|
||
```nickel
|
||
{
|
||
workspace = {
|
||
name = "production",
|
||
version = "1.0.0",
|
||
created = "2025-12-03T14:30:00Z",
|
||
},
|
||
|
||
paths = {
|
||
base = "/opt/workspaces/production",
|
||
infra = "/opt/workspaces/production/infra",
|
||
cache = "/opt/workspaces/production/.cache",
|
||
},
|
||
|
||
providers = {
|
||
active = ["upcloud"],
|
||
default = "upcloud",
|
||
},
|
||
}
|
||
```
|
||
|
||
### Verify Workspace
|
||
|
||
```bash
|
||
# Show workspace info
|
||
provisioning workspace info
|
||
|
||
# List all workspaces
|
||
provisioning workspace list
|
||
|
||
# Show active workspace
|
||
provisioning workspace active
|
||
# Expected: production
|
||
```
|
||
|
||
### View and Validate Workspace Configuration
|
||
|
||
Now you can inspect and validate your Nickel workspace configuration:
|
||
|
||
```bash
|
||
# View complete workspace configuration
|
||
provisioning workspace config show
|
||
|
||
# Show specific workspace
|
||
provisioning workspace config show production
|
||
|
||
# View configuration in different formats
|
||
provisioning workspace config show --format=json
|
||
provisioning workspace config show --format=yaml
|
||
provisioning workspace config show --format=nickel # Raw Nickel file
|
||
|
||
# Validate workspace configuration
|
||
provisioning workspace config validate
|
||
# Output: ✅ Validation complete - all configs are valid
|
||
|
||
# Show configuration hierarchy (priority order)
|
||
provisioning workspace config hierarchy
|
||
```
|
||
|
||
**Configuration Validation**: The Nickel schema automatically validates:
|
||
|
||
- ✅ Semantic versioning format (for example, "1.0.0")
|
||
- ✅ Required sections present (workspace, paths, provisioning, etc.)
|
||
- ✅ Valid file paths and types
|
||
- ✅ Provider configuration exists for active providers
|
||
- ✅ KMS and SOPS settings properly configured
|
||
|
||
---
|
||
|
||
## Step 6: Configure Environment
|
||
|
||
### Set Provider Credentials
|
||
|
||
**UpCloud Provider:**
|
||
|
||
```bash
|
||
# Create provider config
|
||
vim workspace/config/providers/upcloud.toml
|
||
```
|
||
|
||
```toml
|
||
[upcloud]
|
||
username = "your-upcloud-username"
|
||
password = "your-upcloud-password" # Will be encrypted
|
||
|
||
# Default settings
|
||
default_zone = "de-fra1"
|
||
default_plan = "2xCPU-4 GB"
|
||
```
|
||
|
||
**AWS Provider:**
|
||
|
||
```bash
|
||
# Create AWS config
|
||
vim workspace/config/providers/aws.toml
|
||
```
|
||
|
||
```toml
|
||
[aws]
|
||
region = "us-east-1"
|
||
access_key_id = "AKIAXXXXX"
|
||
secret_access_key = "xxxxx" # Will be encrypted
|
||
|
||
# Default settings
|
||
default_instance_type = "t3.medium"
|
||
default_region = "us-east-1"
|
||
```
|
||
|
||
### Encrypt Sensitive Data
|
||
|
||
```bash
|
||
# Generate Age key if not done already
|
||
age-keygen -o ~/.age/key.txt
|
||
|
||
# Encrypt provider configs
|
||
kms encrypt (open workspace/config/providers/upcloud.toml) --backend age \
|
||
| save workspace/config/providers/upcloud.toml.enc
|
||
|
||
# Or use SOPS
|
||
sops --encrypt --age $(cat ~/.age/key.txt | grep "public key:" | cut -d: -f2) \
|
||
workspace/config/providers/upcloud.toml > workspace/config/providers/upcloud.toml.enc
|
||
|
||
# Remove plaintext
|
||
rm workspace/config/providers/upcloud.toml
|
||
```
|
||
|
||
### Configure Local Overrides
|
||
|
||
```bash
|
||
# Edit user-specific settings
|
||
vim workspace/config/local-overrides.toml
|
||
```
|
||
|
||
```toml
|
||
[user]
|
||
name = "admin"
|
||
email = "admin@example.com"
|
||
|
||
[preferences]
|
||
editor = "vim"
|
||
output_format = "yaml"
|
||
confirm_delete = true
|
||
confirm_deploy = true
|
||
|
||
[http]
|
||
use_curl = true # Use curl instead of ureq
|
||
|
||
[paths]
|
||
ssh_key = "~/.ssh/id_ed25519"
|
||
```
|
||
|
||
---
|
||
|
||
## Step 7: Discover and Load Modules
|
||
|
||
### Discover Available Modules
|
||
|
||
```bash
|
||
# Discover task services
|
||
provisioning module discover taskserv
|
||
# Shows: kubernetes, containerd, etcd, cilium, helm, etc.
|
||
|
||
# Discover providers
|
||
provisioning module discover provider
|
||
# Shows: upcloud, aws, local
|
||
|
||
# Discover clusters
|
||
provisioning module discover cluster
|
||
# Shows: buildkit, registry, monitoring, etc.
|
||
```
|
||
|
||
### Load Modules into Workspace
|
||
|
||
```bash
|
||
# Load Kubernetes taskserv
|
||
provisioning module load taskserv production kubernetes
|
||
|
||
# Load multiple modules
|
||
provisioning module load taskserv production kubernetes containerd cilium
|
||
|
||
# Load cluster configuration
|
||
provisioning module load cluster production buildkit
|
||
|
||
# Verify loaded modules
|
||
provisioning module list taskserv production
|
||
provisioning module list cluster production
|
||
```
|
||
|
||
---
|
||
|
||
## Step 8: Validate Configuration
|
||
|
||
Before deploying, validate all configuration:
|
||
|
||
```bash
|
||
# Validate workspace configuration
|
||
provisioning workspace validate
|
||
|
||
# Validate infrastructure configuration
|
||
provisioning validate config
|
||
|
||
# Validate specific infrastructure
|
||
provisioning infra validate --infra production
|
||
|
||
# Check environment variables
|
||
provisioning env
|
||
|
||
# Show all configuration and environment
|
||
provisioning allenv
|
||
```
|
||
|
||
**Expected output:**
|
||
|
||
```plaintext
|
||
✓ Configuration valid
|
||
✓ Provider credentials configured
|
||
✓ Workspace initialized
|
||
✓ Modules loaded: 3 taskservs, 1 cluster
|
||
✓ SSH key configured
|
||
✓ Age encryption key available
|
||
```
|
||
|
||
**Fix any errors** before proceeding to deployment.
|
||
|
||
---
|
||
|
||
## Step 9: Deploy Servers
|
||
|
||
### Preview Server Creation (Dry Run)
|
||
|
||
```bash
|
||
# Check what would be created (no actual changes)
|
||
provisioning server create --infra production --check
|
||
|
||
# With debug output for details
|
||
provisioning server create --infra production --check --debug
|
||
```
|
||
|
||
**Review the output:**
|
||
|
||
- Server names and configurations
|
||
- Zones and regions
|
||
- CPU, memory, disk specifications
|
||
- Estimated costs
|
||
- Network settings
|
||
|
||
### Create Servers
|
||
|
||
```bash
|
||
# Create servers (with confirmation prompt)
|
||
provisioning server create --infra production
|
||
|
||
# Or auto-confirm (skip prompt)
|
||
provisioning server create --infra production --yes
|
||
|
||
# Wait for completion
|
||
provisioning server create --infra production --wait
|
||
```
|
||
|
||
**Expected output:**
|
||
|
||
```plaintext
|
||
Creating servers for infrastructure: production
|
||
|
||
● Creating server: k8s-master-01 (de-fra1, 4xCPU-8 GB)
|
||
● Creating server: k8s-worker-01 (de-fra1, 4xCPU-8 GB)
|
||
● Creating server: k8s-worker-02 (de-fra1, 4xCPU-8 GB)
|
||
|
||
✓ Created 3 servers in 120 seconds
|
||
|
||
Servers:
|
||
• k8s-master-01: 192.168.1.10 (Running)
|
||
• k8s-worker-01: 192.168.1.11 (Running)
|
||
• k8s-worker-02: 192.168.1.12 (Running)
|
||
```
|
||
|
||
### Verify Server Creation
|
||
|
||
```bash
|
||
# List all servers
|
||
provisioning server list --infra production
|
||
|
||
# Show detailed server info
|
||
provisioning server list --infra production --out yaml
|
||
|
||
# SSH to server (test connectivity)
|
||
provisioning server ssh k8s-master-01
|
||
# Type 'exit' to return
|
||
```
|
||
|
||
---
|
||
|
||
## Step 10: Install Task Services
|
||
|
||
Task services are infrastructure components like Kubernetes, databases, monitoring, etc.
|
||
|
||
### Install Kubernetes (Check Mode First)
|
||
|
||
```bash
|
||
# Preview Kubernetes installation
|
||
provisioning taskserv create kubernetes --infra production --check
|
||
|
||
# Shows:
|
||
# - Dependencies required (containerd, etcd)
|
||
# - Configuration to be applied
|
||
# - Resources needed
|
||
# - Estimated installation time
|
||
```
|
||
|
||
### Install Kubernetes
|
||
|
||
```bash
|
||
# Install Kubernetes (with dependencies)
|
||
provisioning taskserv create kubernetes --infra production
|
||
|
||
# Or install dependencies first
|
||
provisioning taskserv create containerd --infra production
|
||
provisioning taskserv create etcd --infra production
|
||
provisioning taskserv create kubernetes --infra production
|
||
|
||
# Monitor progress
|
||
provisioning workflow monitor <task_id>
|
||
```
|
||
|
||
**Expected output:**
|
||
|
||
```plaintext
|
||
Installing taskserv: kubernetes
|
||
|
||
● Installing containerd on k8s-master-01
|
||
● Installing containerd on k8s-worker-01
|
||
● Installing containerd on k8s-worker-02
|
||
✓ Containerd installed (30s)
|
||
|
||
● Installing etcd on k8s-master-01
|
||
✓ etcd installed (20s)
|
||
|
||
● Installing Kubernetes control plane on k8s-master-01
|
||
✓ Kubernetes control plane ready (45s)
|
||
|
||
● Joining worker nodes
|
||
✓ k8s-worker-01 joined (15s)
|
||
✓ k8s-worker-02 joined (15s)
|
||
|
||
✓ Kubernetes installation complete (125 seconds)
|
||
|
||
Cluster Info:
|
||
• Version: 1.28.0
|
||
• Nodes: 3 (1 control-plane, 2 workers)
|
||
• API Server: https://192.168.1.10:6443
|
||
```
|
||
|
||
### Install Additional Services
|
||
|
||
```bash
|
||
# Install Cilium (CNI)
|
||
provisioning taskserv create cilium --infra production
|
||
|
||
# Install Helm
|
||
provisioning taskserv create helm --infra production
|
||
|
||
# Verify all taskservs
|
||
provisioning taskserv list --infra production
|
||
```
|
||
|
||
---
|
||
|
||
## Step 11: Create Clusters
|
||
|
||
Clusters are complete application stacks (for example, BuildKit, OCI Registry, Monitoring).
|
||
|
||
### Create BuildKit Cluster (Check Mode)
|
||
|
||
```bash
|
||
# Preview cluster creation
|
||
provisioning cluster create buildkit --infra production --check
|
||
|
||
# Shows:
|
||
# - Components to be deployed
|
||
# - Dependencies required
|
||
# - Configuration values
|
||
# - Resource requirements
|
||
```
|
||
|
||
### Create BuildKit Cluster
|
||
|
||
```bash
|
||
# Create BuildKit cluster
|
||
provisioning cluster create buildkit --infra production
|
||
|
||
# Monitor deployment
|
||
provisioning workflow monitor <task_id>
|
||
|
||
# Or use plugin for faster monitoring
|
||
orch tasks --status running
|
||
```
|
||
|
||
**Expected output:**
|
||
|
||
```plaintext
|
||
Creating cluster: buildkit
|
||
|
||
● Deploying BuildKit daemon
|
||
● Deploying BuildKit worker
|
||
● Configuring BuildKit cache
|
||
● Setting up BuildKit registry integration
|
||
|
||
✓ BuildKit cluster ready (60 seconds)
|
||
|
||
Cluster Info:
|
||
• BuildKit version: 0.12.0
|
||
• Workers: 2
|
||
• Cache: 50 GB
|
||
• Registry: registry.production.local
|
||
```
|
||
|
||
### Verify Cluster
|
||
|
||
```bash
|
||
# List all clusters
|
||
provisioning cluster list --infra production
|
||
|
||
# Show cluster details
|
||
provisioning cluster list --infra production --out yaml
|
||
|
||
# Check cluster health
|
||
kubectl get pods -n buildkit
|
||
```
|
||
|
||
---
|
||
|
||
## Step 12: Verify Deployment
|
||
|
||
### Comprehensive Health Check
|
||
|
||
```bash
|
||
# Check orchestrator status
|
||
orch status
|
||
# or
|
||
provisioning orchestrator status
|
||
|
||
# Check all servers
|
||
provisioning server list --infra production
|
||
|
||
# Check all taskservs
|
||
provisioning taskserv list --infra production
|
||
|
||
# Check all clusters
|
||
provisioning cluster list --infra production
|
||
|
||
# Verify Kubernetes cluster
|
||
kubectl get nodes
|
||
kubectl get pods --all-namespaces
|
||
```
|
||
|
||
### Run Validation Tests
|
||
|
||
```bash
|
||
# Validate infrastructure
|
||
provisioning infra validate --infra production
|
||
|
||
# Test connectivity
|
||
provisioning server ssh k8s-master-01 "kubectl get nodes"
|
||
|
||
# Test BuildKit
|
||
kubectl exec -it -n buildkit buildkit-0 -- buildctl --version
|
||
```
|
||
|
||
### Expected Results
|
||
|
||
All checks should show:
|
||
|
||
- ✅ Servers: Running
|
||
- ✅ Taskservs: Installed and healthy
|
||
- ✅ Clusters: Deployed and operational
|
||
- ✅ Kubernetes: 3/3 nodes ready
|
||
- ✅ BuildKit: 2/2 workers ready
|
||
|
||
---
|
||
|
||
## Step 13: Post-Deployment
|
||
|
||
### Configure kubectl Access
|
||
|
||
```bash
|
||
# Get kubeconfig from master node
|
||
provisioning server ssh k8s-master-01 "cat ~/.kube/config" > ~/.kube/config-production
|
||
|
||
# Set KUBECONFIG
|
||
export KUBECONFIG=~/.kube/config-production
|
||
|
||
# Verify access
|
||
kubectl get nodes
|
||
kubectl get pods --all-namespaces
|
||
```
|
||
|
||
### Set Up Monitoring (Optional)
|
||
|
||
```bash
|
||
# Deploy monitoring stack
|
||
provisioning cluster create monitoring --infra production
|
||
|
||
# Access Grafana
|
||
kubectl port-forward -n monitoring svc/grafana 3000:80
|
||
# Open: http://localhost:3000
|
||
```
|
||
|
||
### Configure CI/CD Integration (Optional)
|
||
|
||
```bash
|
||
# Generate CI/CD credentials
|
||
provisioning secrets generate aws --ttl 12h
|
||
|
||
# Create CI/CD kubeconfig
|
||
kubectl create serviceaccount ci-cd -n default
|
||
kubectl create clusterrolebinding ci-cd --clusterrole=admin --serviceaccount=default:ci-cd
|
||
```
|
||
|
||
### Backup Configuration
|
||
|
||
```bash
|
||
# Backup workspace configuration
|
||
tar -czf workspace-production-backup.tar.gz workspace/
|
||
|
||
# Encrypt backup
|
||
kms encrypt (open workspace-production-backup.tar.gz | encode base64) --backend age \
|
||
| save workspace-production-backup.tar.gz.enc
|
||
|
||
# Store securely (S3, Vault, etc.)
|
||
```
|
||
|
||
---
|
||
|
||
## Troubleshooting
|
||
|
||
### Server Creation Fails
|
||
|
||
**Problem**: Server creation times out or fails
|
||
|
||
```bash
|
||
# Check provider credentials
|
||
provisioning validate config
|
||
|
||
# Check provider API status
|
||
curl -u username:password https://api.upcloud.com/1.3/account
|
||
|
||
# Try with debug mode
|
||
provisioning server create --infra production --check --debug
|
||
```
|
||
|
||
### Taskserv Installation Fails
|
||
|
||
**Problem**: Kubernetes installation fails
|
||
|
||
```bash
|
||
# Check server connectivity
|
||
provisioning server ssh k8s-master-01
|
||
|
||
# Check logs
|
||
provisioning orchestrator logs | grep kubernetes
|
||
|
||
# Check dependencies
|
||
provisioning taskserv list --infra production | where status == "failed"
|
||
|
||
# Retry installation
|
||
provisioning taskserv delete kubernetes --infra production
|
||
provisioning taskserv create kubernetes --infra production
|
||
```
|
||
|
||
### Plugin Commands Don't Work
|
||
|
||
**Problem**: `auth`, `kms`, or `orch` commands not found
|
||
|
||
```bash
|
||
# Check plugin registration
|
||
plugin list | where name =~ "auth|kms|orch"
|
||
|
||
# Re-register if missing
|
||
cd provisioning/core/plugins/nushell-plugins
|
||
plugin add target/release/nu_plugin_auth
|
||
plugin add target/release/nu_plugin_kms
|
||
plugin add target/release/nu_plugin_orchestrator
|
||
|
||
# Restart Nushell
|
||
exit
|
||
nu
|
||
```
|
||
|
||
### KMS Encryption Fails
|
||
|
||
**Problem**: `kms encrypt` returns error
|
||
|
||
```bash
|
||
# Check backend status
|
||
kms status
|
||
|
||
# Check RustyVault running
|
||
curl http://localhost:8200/v1/sys/health
|
||
|
||
# Use Age backend instead (local)
|
||
kms encrypt "data" --backend age --key age1xxxxxxxxx
|
||
|
||
# Check Age key
|
||
cat ~/.age/key.txt
|
||
```
|
||
|
||
### Orchestrator Not Running
|
||
|
||
**Problem**: `orch status` returns error
|
||
|
||
```bash
|
||
# Check orchestrator status
|
||
ps aux | grep orchestrator
|
||
|
||
# Start orchestrator
|
||
cd provisioning/platform/orchestrator
|
||
./scripts/start-orchestrator.nu --background
|
||
|
||
# Check logs
|
||
tail -f provisioning/platform/orchestrator/data/orchestrator.log
|
||
```
|
||
|
||
### Configuration Validation Errors
|
||
|
||
**Problem**: `provisioning validate config` shows errors
|
||
|
||
```bash
|
||
# Show detailed errors
|
||
provisioning validate config --debug
|
||
|
||
# Check configuration files
|
||
provisioning allenv
|
||
|
||
# Fix missing settings
|
||
vim workspace/config/local-overrides.toml
|
||
```
|
||
|
||
---
|
||
|
||
## Next Steps
|
||
|
||
### Explore Advanced Features
|
||
|
||
1. **Multi-Environment Deployment**
|
||
|
||
```bash
|
||
# Create dev and staging workspaces
|
||
provisioning workspace create dev
|
||
provisioning workspace create staging
|
||
provisioning workspace switch dev
|
||
```
|
||
|
||
1. **Batch Operations**
|
||
|
||
```bash
|
||
# Deploy to multiple clouds
|
||
provisioning batch submit workflows/multi-cloud-deploy.ncl
|
||
```
|
||
|
||
2. **Security Features**
|
||
|
||
```bash
|
||
# Enable MFA
|
||
auth mfa enroll totp
|
||
|
||
# Set up break-glass
|
||
provisioning break-glass request "Emergency access"
|
||
```
|
||
|
||
3. **Compliance and Audit**
|
||
|
||
```bash
|
||
# Generate compliance report
|
||
provisioning compliance report --standard soc2
|
||
```
|
||
|
||
### Learn More
|
||
|
||
- **Quick Reference**: `provisioning sc` or `docs/guides/quickstart-cheatsheet.md`
|
||
- **Update Guide**: `docs/guides/update-infrastructure.md`
|
||
- **Customize Guide**: `docs/guides/customize-infrastructure.md`
|
||
- **Plugin Guide**: `docs/user/PLUGIN_INTEGRATION_GUIDE.md`
|
||
- **Security System**: `docs/architecture/adr-009-security-system-complete.md`
|
||
|
||
### Get Help
|
||
|
||
```bash
|
||
# Show help for any command
|
||
provisioning help
|
||
provisioning help server
|
||
provisioning help taskserv
|
||
|
||
# Check version
|
||
provisioning version
|
||
|
||
# Start Nushell session with provisioning library
|
||
provisioning nu
|
||
```
|
||
|
||
---
|
||
|
||
## Summary
|
||
|
||
You've successfully:
|
||
|
||
✅ Installed Nushell and essential tools
|
||
✅ Built and registered native plugins (10-50x faster operations)
|
||
✅ Cloned and configured the project
|
||
✅ Initialized a production workspace
|
||
✅ Configured provider credentials
|
||
✅ Deployed servers
|
||
✅ Installed Kubernetes and task services
|
||
✅ Created application clusters
|
||
✅ Verified complete deployment
|
||
|
||
**Your infrastructure is now ready for production use!**
|
||
|
||
---
|
||
|
||
**Estimated Total Time**: 30-60 minutes
|
||
**Next Guide**: [Update Infrastructure](update-infrastructure.md)
|
||
**Questions?**: Open an issue or contact <platform-team@example.com>
|
||
|
||
**Last Updated**: 2025-10-09
|
||
**Version**: 3.5.0
|