provisioning/docs/src/development/implementation-guide.md

# Repository Restructuring - Implementation Guide

**Status:** Ready for Implementation
**Estimated Time:** 12-16 days
**Priority:** High
**Related:** [Architecture Analysis](../architecture/repo-dist-analysis.md)

## Overview

This guide provides step-by-step instructions for implementing the repository restructuring and distribution system improvements. Each phase includes specific commands, validation steps, and rollback procedures.

---

## Prerequisites

### Required Tools
- Nushell 0.107.1+
- Rust toolchain (for platform builds)
- Git
- tar/gzip
- curl or wget

### Recommended Tools
- Just (task runner)
- ripgrep (for code searches)
- fd (for file finding)

### Before Starting
1. **Create full backup**
2. **Notify team members**
3. **Create implementation branch**
4. **Set aside dedicated time**

---

## Phase 1: Repository Restructuring (Days 1-4)

### Day 1: Backup and Analysis

#### Step 1.1: Create Complete Backup
```bash
# Create timestamped backup
BACKUP_DIR="/Users/Akasha/project-provisioning-backup-$(date +%Y%m%d)"
cp -r /Users/Akasha/project-provisioning "$BACKUP_DIR"

# Verify backup
ls -lh "$BACKUP_DIR"
du -sh "$BACKUP_DIR"

# Create backup manifest
find "$BACKUP_DIR" -type f > "$BACKUP_DIR/manifest.txt"
echo "✅ Backup created: $BACKUP_DIR"
```

#### Step 1.2: Analyze Current State
```bash
cd /Users/Akasha/project-provisioning

# Count workspace directories
echo "=== Workspace Directories ==="
fd workspace -t d

# Analyze workspace contents
echo "=== Active Workspace ==="
du -sh workspace/

echo "=== Backup Workspaces ==="
du -sh _workspace/ backup-workspace/ workspace-librecloud/

# Find obsolete directories
echo "=== Build Artifacts ==="
du -sh target/ wrks/ NO/

# Save analysis
{
    echo "# Current State Analysis - $(date)"
    echo ""
    echo "## Workspace Directories"
    fd workspace -t d
    echo ""
    echo "## Directory Sizes"
    du -sh workspace/ _workspace/ backup-workspace/ workspace-librecloud/ 2>/dev/null
    echo ""
    echo "## Build Artifacts"
    du -sh target/ wrks/ NO/ 2>/dev/null
} > docs/development/current-state-analysis.txt

echo "✅ Analysis complete: docs/development/current-state-analysis.txt"
```

#### Step 1.3: Identify Dependencies
```bash
# Find all hardcoded paths
echo "=== Hardcoded Paths in Nushell Scripts ==="
rg -t nu "workspace/|_workspace/|backup-workspace/" provisioning/core/nulib/ | tee hardcoded-paths.txt

# Find ENV references (legacy)
echo "=== ENV References ==="
rg "PROVISIONING_" provisioning/core/nulib/ | wc -l

# Find workspace references in configs
echo "=== Config References ==="
rg "workspace" provisioning/config/

echo "✅ Dependencies mapped"
```

#### Step 1.4: Create Implementation Branch
```bash
# Create and switch to implementation branch
git checkout -b feat/repo-restructure

# Commit analysis
git add docs/development/current-state-analysis.txt
git commit -m "docs: add current state analysis for restructuring"

echo "✅ Implementation branch created: feat/repo-restructure"
```

**Validation:**
- ✅ Backup exists and is complete
- ✅ Analysis document created
- ✅ Dependencies mapped
- ✅ Implementation branch ready

---

### Day 2: Directory Restructuring

#### Step 2.1: Create New Directory Structure
```bash
cd /Users/Akasha/project-provisioning

# Create distribution directory structure
mkdir -p distribution/{packages,installers,registry}
echo "✅ Created distribution/"

# Create workspace structure (keep tracked templates)
mkdir -p workspace/{infra,config,extensions,runtime}/{.gitkeep}
mkdir -p workspace/templates/{minimal,kubernetes,multi-cloud}
echo "✅ Created workspace/"

# Verify
tree -L 2 distribution/ workspace/
```

#### Step 2.2: Move Build Artifacts
```bash
# Move Rust build artifacts
if [ -d "target" ]; then
    mv target distribution/target
    echo "✅ Moved target/ to distribution/"
fi

# Move KCL packages
if [ -d "provisioning/tools/dist" ]; then
    mv provisioning/tools/dist/* distribution/packages/ 2>/dev/null || true
    echo "✅ Moved packages to distribution/"
fi

# Move any existing packages
find . -name "*.tar.gz" -o -name "*.zip" | grep -v node_modules | while read pkg; do
    mv "$pkg" distribution/packages/
    echo "  Moved: $pkg"
done
```

#### Step 2.3: Consolidate Workspaces
```bash
# Identify active workspace
echo "=== Current Workspace Status ==="
ls -la workspace/ _workspace/ backup-workspace/ 2>/dev/null

# Interactive workspace consolidation
read -p "Which workspace is currently active? (workspace/_workspace/backup-workspace): " ACTIVE_WS

if [ "$ACTIVE_WS" != "workspace" ]; then
    echo "Consolidating $ACTIVE_WS to workspace/"

    # Merge infra configs
    if [ -d "$ACTIVE_WS/infra" ]; then
        cp -r "$ACTIVE_WS/infra/"* workspace/infra/
    fi

    # Merge configs
    if [ -d "$ACTIVE_WS/config" ]; then
        cp -r "$ACTIVE_WS/config/"* workspace/config/
    fi

    # Merge extensions
    if [ -d "$ACTIVE_WS/extensions" ]; then
        cp -r "$ACTIVE_WS/extensions/"* workspace/extensions/
    fi

    echo "✅ Consolidated workspace"
fi

# Archive old workspace directories
mkdir -p .archived-workspaces
for ws in _workspace backup-workspace workspace-librecloud; do
    if [ -d "$ws" ] && [ "$ws" != "$ACTIVE_WS" ]; then
        mv "$ws" ".archived-workspaces/$(basename $ws)-$(date +%Y%m%d)"
        echo "  Archived: $ws"
    fi
done

echo "✅ Workspaces consolidated"
```

#### Step 2.4: Remove Obsolete Directories
```bash
# Remove build artifacts (already moved)
rm -rf wrks/
echo "✅ Removed wrks/"

# Remove test/scratch directories
rm -rf NO/
echo "✅ Removed NO/"

# Archive presentations (optional)
if [ -d "presentations" ]; then
    read -p "Archive presentations directory? (y/N): " ARCHIVE_PRES
    if [ "$ARCHIVE_PRES" = "y" ]; then
        tar czf presentations-archive-$(date +%Y%m%d).tar.gz presentations/
        rm -rf presentations/
        echo "✅ Archived and removed presentations/"
    fi
fi

# Remove empty directories
find . -type d -empty -delete 2>/dev/null || true

echo "✅ Cleanup complete"
```

#### Step 2.5: Update .gitignore
```bash
# Backup existing .gitignore
cp .gitignore .gitignore.backup

# Update .gitignore
cat >> .gitignore << 'EOF'

# ============================================================================
# Repository Restructure (2025-10-01)
# ============================================================================

# Workspace runtime data (user-specific)
/workspace/infra/
/workspace/config/
/workspace/extensions/
/workspace/runtime/

# Distribution artifacts
/distribution/packages/
/distribution/target/

# Build artifacts
/target/
/provisioning/platform/target/
/provisioning/platform/*/target/

# Rust artifacts
**/*.rs.bk
Cargo.lock

# Archived directories
/.archived-workspaces/

# Temporary files
*.tmp
*.temp
/tmp/
/wrks/
/NO/

# Logs
*.log
/workspace/runtime/logs/

# Cache
.cache/
/workspace/runtime/cache/

# IDE
.vscode/
.idea/
*.swp
*.swo
*~

# OS
.DS_Store
Thumbs.db

# Backup files
*.backup
*.bak

EOF

echo "✅ Updated .gitignore"
```

#### Step 2.6: Commit Restructuring
```bash
# Stage changes
git add -A

# Show what's being committed
git status

# Commit
git commit -m "refactor: restructure repository for clean distribution

- Consolidate workspace directories to single workspace/
- Move build artifacts to distribution/
- Remove obsolete directories (wrks/, NO/)
- Update .gitignore for new structure
- Archive old workspace variants

This is part of Phase 1 of the repository restructuring plan.

Related: docs/architecture/repo-dist-analysis.md"

echo "✅ Restructuring committed"
```

**Validation:**
- ✅ Single `workspace/` directory exists
- ✅ Build artifacts in `distribution/`
- ✅ No `wrks/`, `NO/` directories
- ✅ `.gitignore` updated
- ✅ Changes committed

---

### Day 3: Update Path References

#### Step 3.1: Create Path Update Script
```bash
# Create migration script
cat > provisioning/tools/migration/update-paths.nu << 'EOF'
#!/usr/bin/env nu
# Path update script for repository restructuring

# Find and replace path references
export def main [] {
    print "🔧 Updating path references..."

    let replacements = [
        ["_workspace/" "workspace/"]
        ["backup-workspace/" "workspace/"]
        ["workspace-librecloud/" "workspace/"]
        ["wrks/" "distribution/"]
        ["NO/" "distribution/"]
    ]

    let files = (fd -e nu -e toml -e md . provisioning/)

    mut updated_count = 0

    for file in $files {
        mut content = (open $file)
        mut modified = false

        for replacement in $replacements {
            let old = $replacement.0
            let new = $replacement.1

            if ($content | str contains $old) {
                $content = ($content | str replace -a $old $new)
                $modified = true
            }
        }

        if $modified {
            $content | save -f $file
            $updated_count = $updated_count + 1
            print $"  ✓ Updated: ($file)"
        }
    }

    print $"✅ Updated ($updated_count) files"
}
EOF

chmod +x provisioning/tools/migration/update-paths.nu
```

#### Step 3.2: Run Path Updates
```bash
# Create backup before updates
git stash
git checkout -b feat/path-updates

# Run update script
nu provisioning/tools/migration/update-paths.nu

# Review changes
git diff

# Test a sample file
nu -c "use provisioning/core/nulib/servers/create.nu; print 'OK'"
```

#### Step 3.3: Update CLAUDE.md
```bash
# Update CLAUDE.md with new paths
cat > CLAUDE.md.new << 'EOF'
# CLAUDE.md

[Keep existing content, update paths section...]

## Updated Path Structure (2025-10-01)

### Core System
- **Main CLI**: `provisioning/core/cli/provisioning`
- **Libraries**: `provisioning/core/nulib/`
- **Extensions**: `provisioning/extensions/`
- **Platform**: `provisioning/platform/`

### User Workspace
- **Active Workspace**: `workspace/` (gitignored runtime data)
- **Templates**: `workspace/templates/` (tracked)
- **Infrastructure**: `workspace/infra/` (user configs, gitignored)

### Build System
- **Distribution**: `distribution/` (gitignored artifacts)
- **Packages**: `distribution/packages/`
- **Installers**: `distribution/installers/`

[Continue with rest of content...]
EOF

# Review changes
diff CLAUDE.md CLAUDE.md.new

# Apply if satisfied
mv CLAUDE.md.new CLAUDE.md
```

#### Step 3.4: Update Documentation
```bash
# Find all documentation files
fd -e md . docs/

# Update each doc with new paths
# This is semi-automated - review each file

# Create list of docs to update
fd -e md . docs/ > docs-to-update.txt

# Manual review and update
echo "Review and update each documentation file with new paths"
echo "Files listed in: docs-to-update.txt"
```

#### Step 3.5: Commit Path Updates
```bash
git add -A
git commit -m "refactor: update all path references for new structure

- Update Nushell scripts to use workspace/ instead of variants
- Update CLAUDE.md with new path structure
- Update documentation references
- Add migration script for future path changes

Phase 1.3 of repository restructuring."

echo "✅ Path updates committed"
```

**Validation:**
- ✅ All Nushell scripts reference correct paths
- ✅ CLAUDE.md updated
- ✅ Documentation updated
- ✅ No references to old paths remain

---

### Day 4: Validation and Testing

#### Step 4.1: Automated Validation
```bash
# Create validation script
cat > provisioning/tools/validation/validate-structure.nu << 'EOF'
#!/usr/bin/env nu
# Repository structure validation

export def main [] {
    print "🔍 Validating repository structure..."

    mut passed = 0
    mut failed = 0

    # Check required directories exist
    let required_dirs = [
        "provisioning/core"
        "provisioning/extensions"
        "provisioning/platform"
        "provisioning/kcl"
        "workspace"
        "workspace/templates"
        "distribution"
        "docs"
        "tests"
    ]

    for dir in $required_dirs {
        if ($dir | path exists) {
            print $"  ✓ ($dir)"
            $passed = $passed + 1
        } else {
            print $"  ✗ ($dir) MISSING"
            $failed = $failed + 1
        }
    }

    # Check obsolete directories don't exist
    let obsolete_dirs = [
        "_workspace"
        "backup-workspace"
        "workspace-librecloud"
        "wrks"
        "NO"
    ]

    for dir in $obsolete_dirs {
        if not ($dir | path exists) {
            print $"  ✓ ($dir) removed"
            $passed = $passed + 1
        } else {
            print $"  ✗ ($dir) still exists"
            $failed = $failed + 1
        }
    }

    # Check no old path references
    let old_paths = ["_workspace/" "backup-workspace/" "wrks/"]
    for path in $old_paths {
        let results = (rg -l $path provisioning/ --iglob "!*.md" 2>/dev/null | lines)
        if ($results | is-empty) {
            print $"  ✓ No references to ($path)"
            $passed = $passed + 1
        } else {
            print $"  ✗ Found references to ($path):"
            $results | each { |f| print $"    - ($f)" }
            $failed = $failed + 1
        }
    }

    print ""
    print $"Results: ($passed) passed, ($failed) failed"

    if $failed > 0 {
        error make { msg: "Validation failed" }
    }

    print "✅ Validation passed"
}
EOF

chmod +x provisioning/tools/validation/validate-structure.nu

# Run validation
nu provisioning/tools/validation/validate-structure.nu
```

#### Step 4.2: Functional Testing
```bash
# Test core commands
echo "=== Testing Core Commands ==="

# Version
provisioning/core/cli/provisioning version
echo "✓ version command"

# Help
provisioning/core/cli/provisioning help
echo "✓ help command"

# List
provisioning/core/cli/provisioning list servers
echo "✓ list command"

# Environment
provisioning/core/cli/provisioning env
echo "✓ env command"

# Validate config
provisioning/core/cli/provisioning validate config
echo "✓ validate command"

echo "✅ Functional tests passed"
```

#### Step 4.3: Integration Testing
```bash
# Test workflow system
echo "=== Testing Workflow System ==="

# List workflows
nu -c "use provisioning/core/nulib/workflows/management.nu *; workflow list"
echo "✓ workflow list"

# Test workspace commands
echo "=== Testing Workspace Commands ==="

# Workspace info
provisioning/core/cli/provisioning workspace info
echo "✓ workspace info"

echo "✅ Integration tests passed"
```

#### Step 4.4: Create Test Report
```bash
{
    echo "# Repository Restructuring - Validation Report"
    echo "Date: $(date)"
    echo ""
    echo "## Structure Validation"
    nu provisioning/tools/validation/validate-structure.nu 2>&1
    echo ""
    echo "## Functional Tests"
    echo "✓ version command"
    echo "✓ help command"
    echo "✓ list command"
    echo "✓ env command"
    echo "✓ validate command"
    echo ""
    echo "## Integration Tests"
    echo "✓ workflow list"
    echo "✓ workspace info"
    echo ""
    echo "## Conclusion"
    echo "✅ Phase 1 validation complete"
} > docs/development/phase1-validation-report.md

echo "✅ Test report created: docs/development/phase1-validation-report.md"
```

#### Step 4.5: Update README
```bash
# Update main README with new structure
# This is manual - review and update README.md

echo "📝 Please review and update README.md with new structure"
echo "   - Update directory structure diagram"
echo "   - Update installation instructions"
echo "   - Update quick start guide"
```

#### Step 4.6: Finalize Phase 1
```bash
# Commit validation and reports
git add -A
git commit -m "test: add validation for repository restructuring

- Add structure validation script
- Add functional tests
- Add integration tests
- Create validation report
- Document Phase 1 completion

Phase 1 complete: Repository restructuring validated."

# Merge to implementation branch
git checkout feat/repo-restructure
git merge feat/path-updates

echo "✅ Phase 1 complete and merged"
```

**Validation:**
- ✅ All validation tests pass
- ✅ Functional tests pass
- ✅ Integration tests pass
- ✅ Validation report created
- ✅ README updated
- ✅ Phase 1 changes merged

---

## Phase 2: Build System Implementation (Days 5-8)

### Day 5: Build System Core

#### Step 5.1: Create Build Tools Directory
```bash
mkdir -p provisioning/tools/build
cd provisioning/tools/build

# Create directory structure
mkdir -p {core,platform,extensions,validation,distribution}

echo "✅ Build tools directory created"
```

#### Step 5.2: Implement Core Build System
```bash
# Create main build orchestrator
# See full implementation in repo-dist-analysis.md
# Copy build-system.nu from the analysis document

# Test build system
nu build-system.nu status
```

#### Step 5.3: Implement Core Packaging
```bash
# Create package-core.nu
# This packages Nushell libraries, KCL schemas, templates

# Test core packaging
nu build-system.nu build-core --version dev
```

#### Step 5.4: Create Justfile
```bash
# Create Justfile in project root
# See full Justfile in repo-dist-analysis.md

# Test Justfile
just --list
just status
```

**Validation:**
- ✅ Build system structure exists
- ✅ Core build orchestrator works
- ✅ Core packaging works
- ✅ Justfile functional

### Day 6-8: Continue with Platform, Extensions, and Validation
[Follow similar pattern for remaining build system components]

---

## Phase 3: Installation System (Days 9-11)

### Day 9: Nushell Installer

#### Step 9.1: Create install.nu
```bash
mkdir -p distribution/installers

# Create install.nu
# See full implementation in repo-dist-analysis.md
```

#### Step 9.2: Test Installation
```bash
# Test installation to /tmp
nu distribution/installers/install.nu --prefix /tmp/provisioning-test

# Verify
ls -lh /tmp/provisioning-test/

# Test uninstallation
nu distribution/installers/install.nu uninstall --prefix /tmp/provisioning-test
```

**Validation:**
- ✅ Installer works
- ✅ Files installed to correct locations
- ✅ Uninstaller works
- ✅ No files left after uninstall

---

## Rollback Procedures

### If Phase 1 Fails
```bash
# Restore from backup
rm -rf /Users/Akasha/project-provisioning
cp -r "$BACKUP_DIR" /Users/Akasha/project-provisioning

# Return to main branch
cd /Users/Akasha/project-provisioning
git checkout main
git branch -D feat/repo-restructure
```

### If Build System Fails
```bash
# Revert build system commits
git checkout feat/repo-restructure
git revert <commit-hash>
```

### If Installation Fails
```bash
# Clean up test installation
rm -rf /tmp/provisioning-test
sudo rm -rf /usr/local/lib/provisioning
sudo rm -rf /usr/local/share/provisioning
```

---

## Checklist

### Phase 1: Repository Restructuring
- [ ] Day 1: Backup and analysis complete
- [ ] Day 2: Directory restructuring complete
- [ ] Day 3: Path references updated
- [ ] Day 4: Validation passed

### Phase 2: Build System
- [ ] Day 5: Core build system implemented
- [ ] Day 6: Platform/extensions packaging
- [ ] Day 7: Package validation
- [ ] Day 8: Build system tested

### Phase 3: Installation
- [ ] Day 9: Nushell installer created
- [ ] Day 10: Bash installer and CLI
- [ ] Day 11: Multi-OS testing

### Phase 4: Registry (Optional)
- [ ] Day 12: Registry system
- [ ] Day 13: Registry commands
- [ ] Day 14: Registry hosting

### Phase 5: Documentation
- [ ] Day 15: Documentation updated
- [ ] Day 16: Release prepared

---

## Notes

- **Take breaks between phases** - Don't rush
- **Test thoroughly** - Each phase builds on previous
- **Commit frequently** - Small, atomic commits
- **Document issues** - Track any problems encountered
- **Ask for review** - Get feedback at phase boundaries

---

## Support

If you encounter issues:
1. Check the validation reports
2. Review the rollback procedures
3. Consult the architecture analysis
4. Create an issue in the tracker