2026-01-14 03:09:18 +00:00
|
|
|
# Nickel Package and Module Loader System\n\nThis document describes the package-based architecture implemented for the provisioning system, replacing hardcoded extension paths with a\nflexible module discovery and loading system using Nickel for type-safe configuration.\n\n## Architecture Overview\n\nThe system consists of two main components:\n\n1. **Core Nickel Package**: Distributable core provisioning schemas with type safety\n2. **Module Loader System**: Dynamic discovery and loading of extensions\n\n### Benefits\n\n- **Type-Safe Configuration**: Nickel ensures configuration validity at evaluation time\n- **Clean Separation**: Core package is self-contained and distributable\n- **Plug-and-Play Extensions**: Taskservs, providers, and clusters can be loaded dynamically\n- **Version Management**: Core package and extensions can be versioned independently\n- **Developer Friendly**: Easy workspace setup and module management with lazy evaluation\n\n## Components\n\n### 1. Core Nickel Package (`/provisioning/schemas/`)\n\nContains fundamental schemas for provisioning:\n\n- `main.ncl` - Primary provisioning configuration\n- `server.ncl` - Server definitions and schemas\n- `defaults.ncl` - Default configurations\n- `lib.ncl` - Common library schemas\n- `dependencies.ncl` - Dependency management schemas\n\n**Key Features:**\n\n- No hardcoded extension paths\n- Self-contained and distributable\n- Type-safe package-based imports\n- Lazy evaluation of expensive computations\n\n### 2. Module Discovery System\n\n#### Discovery Commands\n\n```\n# Discover available modules\nmodule-loader discover taskservs # List all taskservs\nmodule-loader discover providers --format yaml # List providers as YAML\nmodule-loader discover clusters redis # Search for redis clusters\n```\n\n#### Supported Module Types\n\n- **Taskservs**: Infrastructure services (kubernetes, redis, postgres, etc.)\n- **Providers**: Cloud providers (upcloud, aws, local)\n- **Clusters**: Complete configurations (buildkit, web, oci-reg)\n\n### 3. Module Loading System\n\n#### Loading Commands\n\n```\n# Load modules into workspace\nmodule-loader load taskservs . [kubernetes, cilium, containerd]\nmodule-loader load providers . [upcloud]\nmodule-loader load clusters . [buildkit]\n\n# Initialize workspace with modules\nmodule-loader init workspace/infra/production \\n --taskservs [kubernetes, cilium] \\n --providers [upcloud]\n```\n\n#### Generated Files\n\n- `taskservs.ncl` - Auto-generated taskserv imports\n- `providers.ncl` - Auto-generated provider imports\n- `clusters.ncl` - Auto-generated cluster imports\n- `.manifest/*.yaml` - Module loading manifests\n\n## Workspace Structure\n\n### New Workspace Layout\n\n```\nworkspace/infra/my-project/\n├── kcl.mod # Package dependencies\n├── servers.ncl # Main server configuration\n├── taskservs.ncl # Auto-generated taskserv imports\n├── providers.ncl # Auto-generated provider imports\n├── clusters.ncl # Auto-generated cluster imports\n├── .taskservs/ # Loaded taskserv modules\n│ ├── kubernetes/\n│ ├── cilium/\n│ └── containerd/\n├── .providers/ # Loaded provider modules\n│ └── upcloud/\n├── .clusters/ # Loaded cluster modules\n│ └── buildkit/\n├── .manifest/ # Module manifests\n│ ├── taskservs.yaml\n│ ├── providers.yaml\n│ └── clusters.yaml\n├── data/ # Runtime data\n├── tmp/ # Temporary files\n├── resources/ # Resource definitions\n└── clusters/ # Cluster configurations\n```\n\n### Import Patterns\n\n#### Before (Old System)\n\n```\n# Hardcoded relative paths\nimport ../../../kcl/server as server\nimport ../../../extensions/taskservs/kubernetes/kcl/kubernetes as k8s\n```\n\n#### After (New System)\n\n```\n# Package-based imports\nimport
|
