provisioning/docs/src/development/provider-development-guide.md

# Cloud Provider Development Guide

**Version**: 2.0
**Status**: Production Ready
**Based On**: Hetzner, UpCloud, AWS (3 completed providers)

---

## Overview: 4-Task Completion Framework

A cloud provider is **production-ready** when it completes all 4 tasks:

| Task | Requirements | Reference |
| ------ | --- | --- |
| **1. Nushell Compliance** | 0 deprecated patterns, full implementations | `provisioning/extensions/providers/hetzner/` |
| **2. Test Infrastructure** | 51 tests (14 unit + 37 integration, mock-based) | `provisioning/extensions/providers/upcloud/tests/` |
| **3. Runtime Templates** | 3+ Jinja2/Bash templates for core resources | `provisioning/extensions/providers/aws/templates/` |
| **4. Nickel Validation** | Schemas pass `nickel typecheck` | `provisioning/extensions/providers/hetzner/nickel/` |

### Execution Sequence

```text
Tarea 4 (5 min) ──────┐
Tarea 1 (main) ───┐   ├──> Tarea 2 (tests)
Tarea 3 (parallel)┘   │
                      └──> Production Ready ✅
```

---

## Nushell 0.109.0+ Core Rules

These rules are **mandatory** for all provider Nushell code:

### Rule 1: Module System & Imports
```nushell
use mod.nu
use api.nu
use servers.nu
```

### Rule 2: Function Signatures
```nushell
def function_name [param: type, optional: type = default] { }
```

### Rule 3: Return Early, Fail Fast
```nushell
def operation [resource: record] {
    if ($resource | get -o id | is-empty) {
        error make {msg: "Resource ID required"}
    }
}
```

### Rule 4: Modern Error Handling (CRITICAL)

**❌ FORBIDDEN** - Deprecated try-catch:
```nushell
try {
    ^external_command
} catch {|err|
    print $"Error: ($err.msg)"
}
```

**✅ REQUIRED** - Modern do/complete pattern:
```nushell
let result = (do { ^external_command } | complete)

if $result.exit_code != 0 {
    error make {msg: $"Command failed: ($result.stderr)"}
}

$result.stdout
```

### Rule 5: Atomic Operations
All operations must fully succeed or fully fail. No partial state changes.

### Rule 12: Structured Error Returns
```nushell
error make {
    msg: "Human-readable message",
    label: {text: "Error context", span: (metadata error).span}
}
```

### Critical Violations (INSTANT FAIL)

❌ **FORBIDDEN**:
- `try { } catch { }` blocks
- `let mut variable = value` (mutable state)
- `error make {msg: "Not implemented"}` (stubs)
- Empty function bodies returning ok
- Deprecated error patterns

---

## Nickel IaC: Three-File Pattern

All Nickel schemas follow this pattern:

### contracts.ncl: Type Definitions

```nickel
{
  Server = {
    id | String,
    name | String,
    instance_type | String,
    zone | String,
  },

  Volume = {
    id | String,
    name | String,
    size | Number,
    type | String,
  }
}
```

### defaults.ncl: Default Values

```nickel
{
  Server = {
    instance_type = "t3.micro",
    zone = "us-east-1a",
  },

  Volume = {
    size = 20,
    type = "gp3",
  }
}
```

### main.ncl: Public API

```nickel
let contracts = import "contracts.ncl" in
let defaults = import "defaults.ncl" in

{
  make_server = fun config => defaults.Server & config,
  make_volume = fun config => defaults.Volume & config,
}
```

### version.ncl: Version Tracking

```nickel
{
  provider_version = "1.0.0",
  cli_tools = {
    hcloud = "1.47.0+",
  },
  nickel_version = "1.7.0+",
}
```

**Validation**:
```bash
nickel typecheck nickel/contracts.ncl
nickel typecheck nickel/defaults.ncl
nickel typecheck nickel/main.ncl
nickel typecheck nickel/version.ncl
nickel export nickel/main.ncl
```

---

## Tarea 1: Nushell Compliance

### Identify Violations

```bash
cd provisioning/extensions/providers/{PROVIDER}

grep -r "try {" nulib/ --include="*.nu" | wc -l
grep -r "let mut " nulib/ --include="*.nu" | wc -l
grep -r "not implemented" nulib/ --include="*.nu" | wc -l
```

All three commands should return `0`.

### Fix Mutable Loops: Accumulation Pattern

```nushell
def retry_with_backoff [
    closure: closure,
    max_attempts: int
]: nothing -> any {
    let result = (
        0..$max_attempts | reduce --fold {
            success: false,
            value: null,
            delay: 100 ms
        } {|attempt, acc|
            if $acc.success {
                $acc
            } else {
                let op_result = (do { $closure | call } | complete)

                if $op_result.exit_code == 0 {
                    {success: true, value: $op_result.stdout, delay: $acc.delay}
                } else if $attempt >= ($max_attempts - 1) {
                    $acc
                } else {
                    sleep $acc.delay
                    {success: false, value: null, delay: ($acc.delay * 2)}
                }
            }
        }
    )

    if $result.success {
        $result.value
    } else {
        error make {msg: $"Failed after ($max_attempts) attempts"}
    }
}
```

### Fix Mutable Loops: Recursive Pattern

```nushell
def _wait_for_state [
    resource_id: string,
    target_state: string,
    timeout_sec: int,
    elapsed: int = 0,
    interval: int = 2
]: nothing -> bool {
    let current = (^aws ec2 describe-volumes \
        --volume-ids $resource_id \
        --query "Volumes[0].State" \
        --output text)

    if ($current | str contains $target_state) {
        true
    } else if $elapsed > $timeout_sec {
        false
    } else {
        sleep ($"($interval)sec" | into duration)
        _wait_for_state $resource_id $target_state $timeout_sec ($elapsed + $interval) $interval
    }
}
```

### Fix Error Handling

```nushell
def create_server [config: record] {
    if ($config | get -o name | is-empty) {
        error make {msg: "Server name required"}
    }

    let api_result = (do {
        ^hcloud server create \
            --name $config.name \
            --type $config.instance_type \
            --format json
    } | complete)

    if $api_result.exit_code != 0 {
        error make {msg: $"Server creation failed: ($api_result.stderr)"}
    }

    let response = ($api_result.stdout | from json)
    {
        id: $response.server.id,
        name: $response.server.name,
        status: "created"
    }
}
```

### Validation

```bash
cd provisioning/extensions/providers/{PROVIDER}

for file in nulib/*/\*.nu; do
    nu --ide-check 100 "$file" 2>&1 | grep -i error && exit 1
done

nu -c "use nulib/{provider}/mod.nu; print 'OK'"

echo "✅ Nushell compliance complete"
```

---

## Tarea 2: Test Infrastructure

### Directory Structure

```text
tests/
├── mocks/
│   └── mock_api_responses.json
├── unit/
│   └── test_utils.nu
├── integration/
│   ├── test_api_client.nu
│   ├── test_server_lifecycle.nu
│   └── test_pricing_cache.nu
└── run_{provider}_tests.nu
```

### Mock API Responses

```json
{
  "list_servers": {
    "servers": [
      {
        "id": "srv-123",
        "name": "test-server",
        "status": "running"
      }
    ]
  },
  "error_401": {
    "error": {"message": "Unauthorized", "code": 401}
  },
  "error_429": {
    "error": {"message": "Rate limited", "code": 429}
  }
}
```

### Unit Tests: 14 Tests

```nushell
def test-result [name: string, result: bool] {
    if $result {
        print $"✓ ($name)"
    } else {
        print $"✗ ($name)"
    }
    $result
}

def test-validate-instance-id [] {
    let valid = "i-1234567890abcdef0"
    let invalid = "invalid-id"

    let test1 = (test-result "Instance ID valid" ($valid | str contains "i-"))
    let test2 = (test-result "Instance ID invalid" (($invalid | str contains "i-") == false))

    $test1 and $test2
}

def test-validate-ipv4 [] {
    let valid = "10.0.1.100"
    let parts = ($valid | split row ".")
    test-result "IPv4 four octets" (($parts | length) == 4)
}

def test-validate-instance-type [] {
    let valid_types = ["t3.micro" "t3.small" "m5.large"]
    let invalid = "invalid_type"

    let test1 = (test-result "Instance type valid" (($valid_types | contains ["t3.micro"])))
    let test2 = (test-result "Instance type invalid" (($valid_types | contains [$invalid]) == false))

    $test1 and $test2
}

def test-validate-zone [] {
    let valid_zones = ["us-east-1a" "us-east-1b" "eu-west-1a"]
    let invalid = "invalid-zone"

    let test1 = (test-result "Zone valid" (($valid_zones | contains ["us-east-1a"])))
    let test2 = (test-result "Zone invalid" (($valid_zones | contains [$invalid]) == false))

    $test1 and $test2
}

def test-validate-volume-id [] {
    let valid = "vol-12345678"
    let invalid = "invalid-vol"

    let test1 = (test-result "Volume ID valid" ($valid | str contains "vol-"))
    let test2 = (test-result "Volume ID invalid" (($invalid | str contains "vol-") == false))

    $test1 and $test2
}

def test-validate-volume-state [] {
    let valid_states = ["available" "in-use" "creating"]
    let invalid = "pending"

    let test1 = (test-result "Volume state valid" (($valid_states | contains ["available"])))
    let test2 = (test-result "Volume state invalid" (($valid_states | contains [$invalid]) == false))

    $test1 and $test2
}

def test-validate-cidr [] {
    let valid = "10.0.0.0/16"
    let invalid = "10.0.0.1"

    let test1 = (test-result "CIDR valid" ($valid | str contains "/"))
    let test2 = (test-result "CIDR invalid" (($invalid | str contains "/") == false))

    $test1 and $test2
}

def test-validate-volume-type [] {
    let valid_types = ["gp2" "gp3" "io1" "io2"]
    let invalid = "invalid-type"

    let test1 = (test-result "Volume type valid" (($valid_types | contains ["gp3"])))
    let test2 = (test-result "Volume type invalid" (($valid_types | contains [$invalid]) == false))

    $test1 and $test2
}

def test-validate-timestamp [] {
    let valid = "2025-01-07T10:00:00.000Z"
    let invalid = "not-a-timestamp"

    let test1 = (test-result "Timestamp valid" ($valid | str contains "T" and $valid | str contains "Z"))
    let test2 = (test-result "Timestamp invalid" (($invalid | str contains "T") == false))

    $test1 and $test2
}

def test-validate-server-state [] {
    let valid_states = ["running" "stopped" "pending"]
    let invalid = "hibernating"

    let test1 = (test-result "Server state valid" (($valid_states | contains ["running"])))
    let test2 = (test-result "Server state invalid" (($valid_states | contains [$invalid]) == false))

    $test1 and $test2
}

def test-validate-security-group [] {
    let valid = "sg-12345678"
    let invalid = "invalid-sg"

    let test1 = (test-result "Security group valid" ($valid | str contains "sg-"))
    let test2 = (test-result "Security group invalid" (($invalid | str contains "sg-") == false))

    $test1 and $test2
}

def test-validate-memory [] {
    let valid_mems = ["512 MB" "1 GB" "2 GB" "4 GB"]
    let invalid = "0 GB"

    let test1 = (test-result "Memory valid" (($valid_mems | contains ["1 GB"])))
    let test2 = (test-result "Memory invalid" (($valid_mems | contains [$invalid]) == false))

    $test1 and $test2
}

def test-validate-vcpu [] {
    let valid_cpus = [1, 2, 4, 8, 16]
    let invalid = 0

    let test1 = (test-result "vCPU valid" (($valid_cpus | contains [1])))
    let test2 = (test-result "vCPU invalid" (($valid_cpus | contains [$invalid]) == false))

    $test1 and $test2
}

def main [] {
    print "=== Unit Tests ==="
    print ""

    let results = [
        (test-validate-instance-id),
        (test-validate-ipv4),
        (test-validate-instance-type),
        (test-validate-zone),
        (test-validate-volume-id),
        (test-validate-volume-state),
        (test-validate-cidr),
        (test-validate-volume-type),
        (test-validate-timestamp),
        (test-validate-server-state),
        (test-validate-security-group),
        (test-validate-memory),
        (test-validate-vcpu)
    ]

    let passed = ($results | where {|it| $it == true} | length)
    let failed = ($results | where {|it| $it == false} | length)

    print ""
    print $"Results: ($passed) passed, ($failed) failed"

    {
        passed: $passed,
        failed: $failed,
        total: ($passed + $failed)
    }
}

main
```

### Integration Tests: 37 Tests across 3 Modules

**Module 1: test_api_client.nu** (13 tests)
- Response structure validation
- Error handling for 401, 404, 429
- Resource listing operations
- Pricing data validation

**Module 2: test_server_lifecycle.nu** (12 tests)
- Server creation, listing, state
- Instance type and zone info
- Storage and security attachment
- Server state transitions

**Module 3: test_pricing_cache.nu** (12 tests)
- Pricing data structure validation
- On-demand vs reserved pricing
- Cost calculations
- Volume pricing operations

### Test Orchestrator

```nushell
def main [] {
    print "=== Provider Test Suite ==="

    let unit_result = (nu tests/unit/test_utils.nu)
    let api_result = (nu tests/integration/test_api_client.nu)
    let lifecycle_result = (nu tests/integration/test_server_lifecycle.nu)
    let pricing_result = (nu tests/integration/test_pricing_cache.nu)

    let total_passed = (
        $unit_result.passed +
        $api_result.passed +
        $lifecycle_result.passed +
        $pricing_result.passed
    )

    let total_failed = (
        $unit_result.failed +
        $api_result.failed +
        $lifecycle_result.failed +
        $pricing_result.failed
    )

    print $"Results: ($total_passed) passed, ($total_failed) failed"

    {
        passed: $total_passed,
        failed: $total_failed,
        success: ($total_failed == 0)
    }
}

let result = (main)
exit (if $result.success {0} else {1})
```

### Validation

```bash
cd provisioning/extensions/providers/{PROVIDER}
nu tests/run_{provider}_tests.nu
```

Expected: 51 tests passing, exit code 0

---

## Tarea 3: Runtime Templates

### Directory Structure

```text
templates/
├── {provider}_servers.j2
├── {provider}_networks.j2
└── {provider}_volumes.j2
```

### Template Example

```jinja2
#!/bin/bash
# {{ provider_name }} Server Provisioning
set -e
{% if debug %}set -x{% endif %}

{%- for server in servers %}
  {%- if server.name %}

echo "Creating server: {{ server.name }}"

{%- if server.instance_type %}
INSTANCE_TYPE="{{ server.instance_type }}"
{%- else %}
INSTANCE_TYPE="t3.micro"
{%- endif %}

SERVER_ID=$(^hcloud server create \
  --name "{{ server.name }}" \
  --type $INSTANCE_TYPE \
  --query 'id' \
  --output text 2>/dev/null)

if [ -z "$SERVER_ID" ]; then
  echo "Failed to create server {{ server.name }}"
  exit 1
fi

echo "✓ Server {{ server.name }} created: $SERVER_ID"

  {%- endif %}
{%- endfor %}

echo "Server provisioning complete"
```

### Validation

```bash
cd provisioning/extensions/providers/{PROVIDER}

for template in templates/*.j2; do
    bash -n <(sed 's/{%.*%}//' "$template" | sed 's/{{.*}}/x/g')
done

echo "✅ Templates valid"
```

---

## Tarea 4: Nickel Schema Validation

```bash
cd provisioning/extensions/providers/{PROVIDER}

nickel typecheck nickel/contracts.ncl || exit 1
nickel typecheck nickel/defaults.ncl || exit 1
nickel typecheck nickel/main.ncl || exit 1
nickel typecheck nickel/version.ncl || exit 1

nickel export nickel/main.ncl || exit 1

echo "✅ Nickel schemas validated"
```

---

## Complete Validation Script

```bash
#!/bin/bash
set -e

PROVIDER="hetzner"
PROV="provisioning/extensions/providers/$PROVIDER"

echo "=== Provider Completeness Check: $PROVIDER ==="

echo ""
echo "✓ Tarea 4: Validating Nickel..."
nickel typecheck "$PROV/nickel/main.ncl"

echo "✓ Tarea 1: Checking Nushell..."
[ $(grep -r "try {" "$PROV/nulib" 2>/dev/null | wc -l) -eq 0 ]
[ $(grep -r "let mut " "$PROV/nulib" 2>/dev/null | wc -l) -eq 0 ]
echo "  - No deprecated patterns ✓"

echo "✓ Tarea 3: Validating templates..."
for f in "$PROV"/templates/*.j2; do
    bash -n <(sed 's/{%.*%}//' "$f" | sed 's/{{.*}}/x/g')
done

echo "✓ Tarea 2: Running tests..."
nu "$PROV/tests/run_${PROVIDER}_tests.nu"

echo ""
echo "╔════════════════════════════════════════╗"
echo "║  ✅ ALL TASKS COMPLETE                 ║"
echo "║     PRODUCTION READY                   ║"
echo "╚════════════════════════════════════════╝"
```

---

## Reference Implementations

- **Hetzner**: `provisioning/extensions/providers/hetzner/`
- **UpCloud**: `provisioning/extensions/providers/upcloud/`
- **AWS**: `provisioning/extensions/providers/aws/`

Use these as templates for new providers.

---

## Quick Start

```bash
cd provisioning/extensions/providers/{PROVIDER}

# Validate completeness
nickel typecheck nickel/main.ncl && \
[ $(grep -r "try {" nulib/ 2>/dev/null | wc -l) -eq 0 ] && \
nu tests/run_{provider}_tests.nu && \
for f in templates/*.j2; do bash -n <(sed 's/{%.*%}//' "$f"); done && \
echo "✅ PRODUCTION READY"
```