# Taskserv Validation and Testing Guide **Version**: 1.0.0 **Date**: 2025-10-06 **Status**: Production Ready --- ## Overview The taskserv validation and testing system provides comprehensive evaluation of infrastructure services before deployment, reducing errors and increasing confidence in deployments. ## Validation Levels ### 1. Static Validation Validates configuration files, templates, and scripts without requiring infrastructure access. **What it checks:** - KCL schema syntax and semantics - Jinja2 template syntax - Shell script syntax (with shellcheck if available) - File structure and naming conventions **Command:** ``` provisioning taskserv validate kubernetes --level static ``` ### 2. Dependency Validation Checks taskserv dependencies, conflicts, and requirements. **What it checks:** - Required dependencies are available - Optional dependencies status - Conflicting taskservs - Resource requirements (memory, CPU, disk) - Health check configuration **Command:** ``` provisioning taskserv validate kubernetes --level dependencies ``` **Check against infrastructure:** ``` provisioning taskserv check-deps kubernetes --infra my-project ``` ### 3. Check Mode (Dry-Run) Enhanced check mode that performs validation and previews deployment without making changes. **What it does:** - Runs static validation - Validates dependencies - Previews configuration generation - Lists files to be deployed - Checks prerequisites (without SSH in check mode) **Command:** ``` provisioning taskserv create kubernetes --check ``` ### 4. Sandbox Testing Tests taskserv in isolated container environment before actual deployment. **What it tests:** - Package prerequisites - Configuration validity - Script execution - Health check simulation **Command:** ``` # Test with Docker provisioning taskserv test kubernetes --runtime docker # Test with Podman provisioning taskserv test kubernetes --runtime podman # Keep container for inspection provisioning taskserv test kubernetes --runtime docker --keep ``` --- ## Complete Validation Workflow ### Recommended Validation Sequence ``` # 1. Static validation (fastest, no infrastructure needed) provisioning taskserv validate kubernetes --level static -v # 2. Dependency validation provisioning taskserv check-deps kubernetes --infra my-project # 3. Check mode (dry-run with full validation) provisioning taskserv create kubernetes --check -v # 4. Sandbox testing (optional, requires Docker/Podman) provisioning taskserv test kubernetes --runtime docker # 5. Actual deployment (after all validations pass) provisioning taskserv create kubernetes ``` ### Quick Validation (All Levels) ``` # Run all validation levels provisioning taskserv validate kubernetes --level all -v ``` --- ## Validation Commands Reference ### `provisioning taskserv validate ` Multi-level validation framework. **Options:** - `--level ` - Validation level: static, dependencies, health, all (default: all) - `--infra ` - Infrastructure context - `--settings ` - Settings file path - `--verbose` - Verbose output - `--out ` - Output format: json, yaml, text **Examples:** ``` # Complete validation provisioning taskserv validate kubernetes # Only static validation provisioning taskserv validate kubernetes --level static # With verbose output provisioning taskserv validate kubernetes -v # JSON output provisioning taskserv validate kubernetes --out json ``` ### `provisioning taskserv check-deps ` Check dependencies against infrastructure. **Options:** - `--infra ` - Infrastructure context - `--settings ` - Settings file path - `--verbose` - Verbose output **Examples:** ``` # Check dependencies provisioning taskserv check-deps kubernetes --infra my-project # Verbose output provisioning taskserv check-deps kubernetes --infra my-project -v ``` ### `provisioning taskserv create --check` Enhanced check mode with full validation and preview. **Options:** - `--check` - Enable check mode (no actual deployment) - `--verbose` - Verbose output - All standard create options **Examples:** ``` # Check mode with verbose output provisioning taskserv create kubernetes --check -v # Check specific server provisioning taskserv create kubernetes server-01 --check ``` ### `provisioning taskserv test ` Sandbox testing in isolated environment. **Options:** - `--runtime ` - Runtime: docker, podman, native (default: docker) - `--infra ` - Infrastructure context - `--settings ` - Settings file path - `--keep` - Keep container after test - `--verbose` - Verbose output **Examples:** ``` # Test with Docker provisioning taskserv test kubernetes --runtime docker # Test with Podman provisioning taskserv test kubernetes --runtime podman # Keep container for debugging provisioning taskserv test kubernetes --keep -v # Connect to kept container docker exec -it taskserv-test-kubernetes bash ``` --- ## Validation Output ### Static Validation ``` Taskserv Validation Taskserv: kubernetes Level: static Validating Nickel schemas for kubernetes... Checking main.ncl... ✓ Valid Checking version.ncl... ✓ Valid Checking dependencies.ncl... ✓ Valid Validating templates for kubernetes... Checking env-kubernetes.j2... ✓ Basic syntax OK Checking install-kubernetes.sh... ✓ Basic syntax OK Validation Summary ✓ nickel: 0 errors, 0 warnings ✓ templates: 0 errors, 0 warnings ✓ scripts: 0 errors, 0 warnings Overall Status ✓ VALID - 0 warnings ``` ### Dependency Validation ``` Dependency Validation Report Taskserv: kubernetes Status: VALID Required Dependencies: • containerd • etcd • os Optional Dependencies: • cilium • helm Conflicts: • docker • podman ``` ### Check Mode Output ``` Check Mode: kubernetes on server-01 → Running static validation... ✓ Static validation passed → Checking dependencies... ✓ Dependencies OK Required: containerd, etcd, os → Previewing configuration generation... ✓ Configuration preview generated Files to process: 15 → Checking prerequisites... ℹ Prerequisite checks (preview mode): ⊘ Server accessibility: Check mode - SSH not tested ℹ Directory /tmp: Would verify directory exists ℹ Command bash: Would verify command is available Check Mode Summary ✓ All validations passed 💡 Taskserv can be deployed with: provisioning taskserv create kubernetes ``` ### Test Output ``` Taskserv Sandbox Testing Taskserv: kubernetes Runtime: docker → Running pre-test validation... ✓ Validation passed → Preparing sandbox environment... Using base image: ubuntu:22.04 ✓ Sandbox prepared: a1b2c3d4e5f6 → Running tests in sandbox... Test 1: Package prerequisites... Test 2: Configuration validity... Test 3: Script execution... Test 4: Health check simulation... Test Summary Total tests: 4 Passed: 4 Failed: 0 Skipped: 0 Detailed Results: ✓ Package prerequisites: Package manager accessible ✓ Configuration validity: 3 configuration files validated ✓ Script execution: 2 scripts validated ✓ Health check: Health check configuration valid: http://localhost:6443/healthz ✓ All tests passed ``` --- ## Integration with CI/CD ### GitLab CI Example ``` validate-taskservs: stage: validate script: - provisioning taskserv validate kubernetes --level all --out json - provisioning taskserv check-deps kubernetes --infra production test-taskservs: stage: test script: - provisioning taskserv test kubernetes --runtime docker dependencies: - validate-taskservs deploy-taskservs: stage: deploy script: - provisioning taskserv create kubernetes dependencies: - test-taskservs only: - main ``` ### GitHub Actions Example ``` name: Taskserv Validation on: [push, pull_request] jobs: validate: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Validate Taskservs run: | provisioning taskserv validate kubernetes --level all -v - name: Check Dependencies run: | provisioning taskserv check-deps kubernetes --infra production - name: Test in Sandbox run: | provisioning taskserv test kubernetes --runtime docker ``` --- ## Troubleshooting ### shellcheck not found If shellcheck is not available, script validation will be skipped with a warning. **Install shellcheck:** ``` # macOS brew install shellcheck # Ubuntu/Debian apt install shellcheck # Fedora dnf install shellcheck ``` ### Docker/Podman not available Sandbox testing requires Docker or Podman. **Check runtime:** ``` # Docker docker ps # Podman podman ps # Use native mode (limited testing) provisioning taskserv test kubernetes --runtime native ``` ### Nickel type checking errors Nickel type checking errors indicate syntax or type problems. **Common fixes:** - Check schema syntax in `.ncl` files - Validate imports and dependencies - Run `nickel format` to format files - Check `manifest.toml` dependencies ### Dependency conflicts If conflicting taskservs are detected: - Remove conflicting taskserv first - Check infrastructure configuration - Review dependency declarations in `dependencies.ncl` --- ## Advanced Usage ### Custom Validation Scripts You can create custom validation scripts by extending the validation framework: ``` # custom_validation.nu use provisioning/core/nulib/taskservs/validate.nu * def custom-validate [taskserv: string] { # Custom validation logic let result = (validate-nickel-schemas $taskserv --verbose=true) # Additional custom checks # ... return $result } ``` ### Batch Validation Validate multiple taskservs: ``` # Validate all taskservs in infrastructure for taskserv in (provisioning taskserv list | get name) { provisioning taskserv validate $taskserv } ``` ### Automated Testing Create test suite for all taskservs: ``` #!/usr/bin/env nu let taskservs = ["kubernetes", "containerd", "cilium", "etcd"] for ts in $taskservs { print $"Testing ($ts)..." provisioning taskserv test $ts --runtime docker } ``` --- ## Best Practices ### Before Deployment 1. **Always validate** before deploying to production 2. **Run check mode** to preview changes 3. **Test in sandbox** for critical services 4. **Check dependencies** in infrastructure context ### During Development 1. **Validate frequently** during taskserv development 2. **Use verbose mode** to understand validation details 3. **Fix warnings** even if validation passes 4. **Keep containers** for debugging test failures ### In CI/CD 1. **Fail fast** on validation errors 2. **Require all tests pass** before merge 3. **Generate reports** in JSON format for analysis 4. **Archive test results** for audit trail --- ## Related Documentation - [Taskserv Development Guide](taskserv-development-guide.md) - KCL Schema Reference - [Dependency Management](dependency-management.md) - [CI/CD Integration](cicd-integration.md) --- ## Version History | Version | Date | Changes | | --------- | ------ | --------- | | 1.0.0 | 2025-10-06 | Initial validation and testing guide | --- **Maintained By**: Infrastructure Team **Review Cycle**: Quarterly