# Taskserv Validation and Testing Guide\n\n**Version**: 1.0.0\n**Date**: 2025-10-06\n**Status**: Production Ready\n\n---\n\n## Overview\n\nThe taskserv validation and testing system provides comprehensive evaluation of infrastructure services before deployment, reducing errors and\nincreasing confidence in deployments.\n\n## Validation Levels\n\n### 1. Static Validation\n\nValidates configuration files, templates, and scripts without requiring infrastructure access.\n\n**What it checks:**\n\n- KCL schema syntax and semantics\n- Jinja2 template syntax\n- Shell script syntax (with shellcheck if available)\n- File structure and naming conventions\n\n**Command:**\n\n```\nprovisioning taskserv validate kubernetes --level static\n```\n\n### 2. Dependency Validation\n\nChecks taskserv dependencies, conflicts, and requirements.\n\n**What it checks:**\n\n- Required dependencies are available\n- Optional dependencies status\n- Conflicting taskservs\n- Resource requirements (memory, CPU, disk)\n- Health check configuration\n\n**Command:**\n\n```\nprovisioning taskserv validate kubernetes --level dependencies\n```\n\n**Check against infrastructure:**\n\n```\nprovisioning taskserv check-deps kubernetes --infra my-project\n```\n\n### 3. Check Mode (Dry-Run)\n\nEnhanced check mode that performs validation and previews deployment without making changes.\n\n**What it does:**\n\n- Runs static validation\n- Validates dependencies\n- Previews configuration generation\n- Lists files to be deployed\n- Checks prerequisites (without SSH in check mode)\n\n**Command:**\n\n```\nprovisioning taskserv create kubernetes --check\n```\n\n### 4. Sandbox Testing\n\nTests taskserv in isolated container environment before actual deployment.\n\n**What it tests:**\n\n- Package prerequisites\n- Configuration validity\n- Script execution\n- Health check simulation\n\n**Command:**\n\n```\n# Test with Docker\nprovisioning taskserv test kubernetes --runtime docker\n\n# Test with Podman\nprovisioning taskserv test kubernetes --runtime podman\n\n# Keep container for inspection\nprovisioning taskserv test kubernetes --runtime docker --keep\n```\n\n---\n\n## Complete Validation Workflow\n\n### Recommended Validation Sequence\n\n```\n# 1. Static validation (fastest, no infrastructure needed)\nprovisioning taskserv validate kubernetes --level static -v\n\n# 2. Dependency validation\nprovisioning taskserv check-deps kubernetes --infra my-project\n\n# 3. Check mode (dry-run with full validation)\nprovisioning taskserv create kubernetes --check -v\n\n# 4. Sandbox testing (optional, requires Docker/Podman)\nprovisioning taskserv test kubernetes --runtime docker\n\n# 5. Actual deployment (after all validations pass)\nprovisioning taskserv create kubernetes\n```\n\n### Quick Validation (All Levels)\n\n```\n# Run all validation levels\nprovisioning taskserv validate kubernetes --level all -v\n```\n\n---\n\n## Validation Commands Reference\n\n### `provisioning taskserv validate <taskserv>`\n\nMulti-level validation framework.\n\n**Options:**\n\n- `--level <level>` - Validation level: static, dependencies, health, all (default: all)\n- `--infra <name>` - Infrastructure context\n- `--settings <path>` - Settings file path\n- `--verbose` - Verbose output\n- `--out <format>` - Output format: json, yaml, text\n\n**Examples:**\n\n```\n# Complete validation\nprovisioning taskserv validate kubernetes\n\n# Only static validation\nprovisioning taskserv validate kubernetes --level static\n\n# With verbose output\nprovisioning taskserv validate kubernetes -v\n\n# JSON output\nprovisioning taskserv validate kubernetes --out json\n```\n\n### `provisioning taskserv check-deps <taskserv>`\n\nCheck dependencies against infrastructure.\n\n**Options:**\n\n- `--infra <name>` - Infrastructure context\n- `--settings <path>` - Settings file path\n- `--verbose` - Verbose output\n\n**Examples:**\n\n```\n# Check dependencies\nprovisioning taskserv check-deps kubernetes --infra my-project\n\n# Verbose output\nprovisioning taskserv check-deps kubernetes --infra my-project -v\n```\n\n### `provisioning taskserv create <taskserv> --check`\n\nEnhanced check mode with full validation and preview.\n\n**Options:**\n\n- `--check` - Enable check mode (no actual deployment)\n- `--verbose` - Verbose output\n- All standard create options\n\n**Examples:**\n\n```\n# Check mode with verbose output\nprovisioning taskserv create kubernetes --check -v\n\n# Check specific server\nprovisioning taskserv create kubernetes server-01 --check\n```\n\n### `provisioning taskserv test <taskserv>`\n\nSandbox testing in isolated environment.\n\n**Options:**\n\n- `--runtime <name>` - Runtime: docker, podman, native (default: docker)\n- `--infra <name>` - Infrastructure context\n- `--settings <path>` - Settings file path\n- `--keep` - Keep container after test\n- `--verbose` - Verbose output\n\n**Examples:**\n\n```\n# Test with Docker\nprovisioning taskserv test kubernetes --runtime docker\n\n# Test with Podman\nprovisioning taskserv test kubernetes --runtime podman\n\n# Keep container for debugging\nprovisioning taskserv test kubernetes --keep -v\n\n# Connect to kept container\ndocker exec -it taskserv-test-kubernetes bash\n```\n\n---\n\n## Validation Output\n\n### Static Validation\n\n```\nTaskserv Validation\nTaskserv: kubernetes\nLevel: static\n\nValidating Nickel schemas for kubernetes...\n  Checking main.ncl...\n    ✓ Valid\n  Checking version.ncl...\n    ✓ Valid\n  Checking dependencies.ncl...\n    ✓ Valid\n\nValidating templates for kubernetes...\n  Checking env-kubernetes.j2...\n    ✓ Basic syntax OK\n  Checking install-kubernetes.sh...\n    ✓ Basic syntax OK\n\nValidation Summary\n✓ nickel: 0 errors, 0 warnings\n✓ templates: 0 errors, 0 warnings\n✓ scripts: 0 errors, 0 warnings\n\nOverall Status\n✓ VALID - 0 warnings\n```\n\n### Dependency Validation\n\n```\nDependency Validation Report\nTaskserv: kubernetes\n\nStatus: VALID\n\nRequired Dependencies:\n  • containerd\n  • etcd\n  • os\n\nOptional Dependencies:\n  • cilium\n  • helm\n\nConflicts:\n  • docker\n  • podman\n```\n\n### Check Mode Output\n\n```\nCheck Mode: kubernetes on server-01\n\n→ Running static validation...\n  ✓ Static validation passed\n\n→ Checking dependencies...\n  ✓ Dependencies OK\n    Required: containerd, etcd, os\n\n→ Previewing configuration generation...\n  ✓ Configuration preview generated\n    Files to process: 15\n\n→ Checking prerequisites...\n  ℹ Prerequisite checks (preview mode):\n    ⊘ Server accessibility: Check mode - SSH not tested\n    ℹ Directory /tmp: Would verify directory exists\n    ℹ Command bash: Would verify command is available\n\nCheck Mode Summary\n✓ All validations passed\n\n💡 Taskserv can be deployed with: provisioning taskserv create kubernetes\n```\n\n### Test Output\n\n```\nTaskserv Sandbox Testing\nTaskserv: kubernetes\nRuntime: docker\n\n→ Running pre-test validation...\n✓ Validation passed\n\n→ Preparing sandbox environment...\n  Using base image: ubuntu:22.04\n✓ Sandbox prepared: a1b2c3d4e5f6\n\n→ Running tests in sandbox...\n  Test 1: Package prerequisites...\n  Test 2: Configuration validity...\n  Test 3: Script execution...\n  Test 4: Health check simulation...\n\nTest Summary\nTotal tests: 4\nPassed: 4\nFailed: 0\nSkipped: 0\n\nDetailed Results:\n  ✓ Package prerequisites: Package manager accessible\n  ✓ Configuration validity: 3 configuration files validated\n  ✓ Script execution: 2 scripts validated\n  ✓ Health check: Health check configuration valid: http://localhost:6443/healthz\n\n✓ All tests passed\n```\n\n---\n\n## Integration with CI/CD\n\n### GitLab CI Example\n\n```\nvalidate-taskservs:\n  stage: validate\n  script:\n    - provisioning taskserv validate kubernetes --level all --out json\n    - provisioning taskserv check-deps kubernetes --infra production\n\ntest-taskservs:\n  stage: test\n  script:\n    - provisioning taskserv test kubernetes --runtime docker\n  dependencies:\n    - validate-taskservs\n\ndeploy-taskservs:\n  stage: deploy\n  script:\n    - provisioning taskserv create kubernetes\n  dependencies:\n    - test-taskservs\n  only:\n    - main\n```\n\n### GitHub Actions Example\n\n```\nname: Taskserv Validation\n\non: [push, pull_request]\n\njobs:\n  validate:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v3\n\n      - name: Validate Taskservs\n        run: |\n          provisioning taskserv validate kubernetes --level all -v\n\n      - name: Check Dependencies\n        run: |\n          provisioning taskserv check-deps kubernetes --infra production\n\n      - name: Test in Sandbox\n        run: |\n          provisioning taskserv test kubernetes --runtime docker\n```\n\n---\n\n## Troubleshooting\n\n### shellcheck not found\n\nIf shellcheck is not available, script validation will be skipped with a warning.\n\n**Install shellcheck:**\n\n```\n# macOS\nbrew install shellcheck\n\n# Ubuntu/Debian\napt install shellcheck\n\n# Fedora\ndnf install shellcheck\n```\n\n### Docker/Podman not available\n\nSandbox testing requires Docker or Podman.\n\n**Check runtime:**\n\n```\n# Docker\ndocker ps\n\n# Podman\npodman ps\n\n# Use native mode (limited testing)\nprovisioning taskserv test kubernetes --runtime native\n```\n\n### Nickel type checking errors\n\nNickel type checking errors indicate syntax or type problems.\n\n**Common fixes:**\n\n- Check schema syntax in `.ncl` files\n- Validate imports and dependencies\n- Run `nickel format` to format files\n- Check `manifest.toml` dependencies\n\n### Dependency conflicts\n\nIf conflicting taskservs are detected:\n\n- Remove conflicting taskserv first\n- Check infrastructure configuration\n- Review dependency declarations in `dependencies.ncl`\n\n---\n\n## Advanced Usage\n\n### Custom Validation Scripts\n\nYou can create custom validation scripts by extending the validation framework:\n\n```\n# custom_validation.nu\nuse provisioning/core/nulib/taskservs/validate.nu *\n\ndef custom-validate [taskserv: string] {\n    # Custom validation logic\n    let result = (validate-nickel-schemas $taskserv --verbose=true)\n\n    # Additional custom checks\n    # ...\n\n    return $result\n}\n```\n\n### Batch Validation\n\nValidate multiple taskservs:\n\n```\n# Validate all taskservs in infrastructure\nfor taskserv in (provisioning taskserv list | get name) {\n    provisioning taskserv validate $taskserv\n}\n```\n\n### Automated Testing\n\nCreate test suite for all taskservs:\n\n```\n#!/usr/bin/env nu\n\nlet taskservs = ["kubernetes", "containerd", "cilium", "etcd"]\n\nfor ts in $taskservs {\n    print $"Testing ($ts)..."\n    provisioning taskserv test $ts --runtime docker\n}\n```\n\n---\n\n## Best Practices\n\n### Before Deployment\n\n1. **Always validate** before deploying to production\n2. **Run check mode** to preview changes\n3. **Test in sandbox** for critical services\n4. **Check dependencies** in infrastructure context\n\n### During Development\n\n1. **Validate frequently** during taskserv development\n2. **Use verbose mode** to understand validation details\n3. **Fix warnings** even if validation passes\n4. **Keep containers** for debugging test failures\n\n### In CI/CD\n\n1. **Fail fast** on validation errors\n2. **Require all tests pass** before merge\n3. **Generate reports** in JSON format for analysis\n4. **Archive test results** for audit trail\n\n---\n\n## Related Documentation\n\n- [Taskserv Development Guide](taskserv-development-guide.md)\n- KCL Schema Reference\n- [Dependency Management](dependency-management.md)\n- [CI/CD Integration](cicd-integration.md)\n\n---\n\n## Version History\n\n| Version | Date | Changes |\n| --------- | ------ | --------- |\n| 1.0.0 | 2025-10-06 | Initial validation and testing guide |\n\n---\n\n**Maintained By**: Infrastructure Team\n**Review Cycle**: Quarterly