updates for deployment tool

This commit is contained in:
Geir Okkenhaug Jerstad 2025-06-15 11:01:41 +02:00
parent 9f7c2640b5
commit 5332351a06
4 changed files with 41 additions and 106 deletions

View file

@ -1,29 +1,35 @@
# Deploy-rs Integration Summary # Deploy-rs Integration Summary
## Overview ## Overview
Successfully integrated deploy-rs into the Home Lab infrastructure as a modern, production-ready deployment method alongside the existing shell script approach. Successfully integrated deploy-rs into the Home Lab infrastructure as a modern, production-ready deployment method alongside the existing shell script approach.
## Completed Tasks ✅ ## Completed Tasks ✅
### Task 1: Add deploy-rs input to flake.nix ✅ ### Task 1: Add deploy-rs input to flake.nix ✅
- Added `deploy-rs.url = "github:serokell/deploy-rs"` to flake inputs - Added `deploy-rs.url = "github:serokell/deploy-rs"` to flake inputs
- Exposed deploy-rs in outputs function parameters - Exposed deploy-rs in outputs function parameters
- Validated with `nix flake check` - Validated with `nix flake check`
### Task 2: Create basic deploy-rs configuration ✅ ### Task 2: Create basic deploy-rs configuration ✅
- Configured all 4 machines in `deploy.nodes` section - Configured all 4 machines in `deploy.nodes` section
- Used Tailscale hostnames for reliable connectivity - Used Tailscale hostnames for reliable connectivity
- Set up proper SSH users and activation paths - Set up proper SSH users and activation paths
### Task 3: Add deploy-rs health checks ✅ ### Task 3: Add deploy-rs health checks ✅
- Configured activation timeouts: 180s (local), 240s (VPS) - Configured activation timeouts: 180s (local), 240s (VPS)
- Set confirm timeouts: 30s for all machines - Set confirm timeouts: 30s for all machines
- Enabled autoRollback and magicRollback for safety - Enabled autoRollback and magicRollback for safety
### Task 4: Test deploy-rs on sleeper-service ✅ ### Task 4: Test deploy-rs on sleeper-service ✅
**Status**: Successfully completed on June 15, 2025 **Status**: Successfully completed on June 15, 2025
**Results**: **Results**:
- ✅ Dry-run deployment successful - ✅ Dry-run deployment successful
- ✅ Actual deployment successful - ✅ Actual deployment successful
- ✅ Service management (transmission.service restart) - ✅ Service management (transmission.service restart)
@ -33,14 +39,17 @@ Successfully integrated deploy-rs into the Home Lab infrastructure as a modern,
- ✅ Tailscale connectivity working perfectly - ✅ Tailscale connectivity working perfectly
### Task 5: Integrate deploy-rs with lab tool ✅ ### Task 5: Integrate deploy-rs with lab tool ✅
**Status**: Successfully completed on June 15, 2025 **Status**: Successfully completed on June 15, 2025
**New Commands Added**: **New Commands Added**:
- `lab deploy-rs <machine> [--dry-run]` - Modern deployment with automatic rollback - `lab deploy-rs <machine> [--dry-run]` - Modern deployment with automatic rollback
- `lab update-flake` - Update package versions and validate configuration - `lab update-flake` - Update package versions and validate configuration
- `lab hybrid-update [target] [--dry-run]` - Combined flake update + deploy-rs deployment - `lab hybrid-update [target] [--dry-run]` - Combined flake update + deploy-rs deployment
**Features**: **Features**:
- Hybrid approach combining package updates with deployment safety - Hybrid approach combining package updates with deployment safety
- Maintains existing legacy deployment commands for compatibility - Maintains existing legacy deployment commands for compatibility
- Comprehensive help documentation with examples - Comprehensive help documentation with examples
@ -60,6 +69,7 @@ Successfully integrated deploy-rs into the Home Lab infrastructure as a modern,
## Usage Examples ## Usage Examples
### Basic Deploy-rs Usage ### Basic Deploy-rs Usage
```bash ```bash
# Deploy with automatic rollback protection # Deploy with automatic rollback protection
lab deploy-rs sleeper-service lab deploy-rs sleeper-service
@ -69,6 +79,7 @@ lab deploy-rs sleeper-service --dry-run
``` ```
### Hybrid Update Usage (Recommended) ### Hybrid Update Usage (Recommended)
```bash ```bash
# Update packages and deploy to specific machine # Update packages and deploy to specific machine
lab hybrid-update sleeper-service lab hybrid-update sleeper-service
@ -82,6 +93,7 @@ lab update-flake
``` ```
### Legacy Usage (Still Available) ### Legacy Usage (Still Available)
```bash ```bash
# Traditional deployment method # Traditional deployment method
lab deploy sleeper-service boot lab deploy sleeper-service boot
@ -91,6 +103,7 @@ lab update boot
## Technical Implementation ## Technical Implementation
### Deploy-rs Configuration ### Deploy-rs Configuration
```nix ```nix
deploy.nodes = { deploy.nodes = {
sleeper-service = { sleeper-service = {
@ -112,30 +125,25 @@ deploy.nodes = {
``` ```
### Lab Tool Integration ### Lab Tool Integration
The lab tool now provides three deployment approaches: The lab tool now provides three deployment approaches:
1. **Legacy**: Reliable SSH + rsync method (existing workflow) 1. **Legacy**: Reliable SSH + rsync method (existing workflow)
2. **Modern**: Direct deploy-rs usage with safety features 2. **Modern**: Direct deploy-rs usage with safety features
3. **Hybrid**: Automated package updates + deploy-rs deployment 3. **Hybrid**: Automated package updates + deploy-rs deployment
## Pending Tasks ## Next Steps
### Completed Tasks ✅ ### Pending Tasks
- ✅ **Task 6**: Test deploy-rs on all machines (grey-area, reverse-proxy, congenital-optimist) - **COMPLETED**
**Results:** - **Task 6**: Test deploy-rs on all machines (grey-area, reverse-proxy, congenital-optimist)
- **grey-area**: ✅ Deploy-rs deployment successful (both dry-run and actual)
- **reverse-proxy**: ✅ Deploy-rs deployment successful (dry-run completed)
- **congenital-optimist**: ✅ Deploy-rs deployment successful (both dry-run and actual)
- **Infrastructure improvements**: Added `sma` user to local machine, created shared shell aliases module
- **User management**: Resolved shell alias conflicts with user-specific aliases
### Remaining Tasks
- **Task 7**: Add deploy-rs status monitoring to lab tool - **Task 7**: Add deploy-rs status monitoring to lab tool
- **Task 8**: Create deployment workflow documentation - **Task 8**: Create deployment workflow documentation
- **Task 9**: Optimize deploy-rs for home lab network - **Task 9**: Optimize deploy-rs for home lab network
- **Task 10**: Implement emergency rollback procedures - **Task 10**: Implement emergency rollback procedures
### Recommendations ### Recommendations
1. Use **hybrid-update** for regular maintenance (combines updates + safety) 1. Use **hybrid-update** for regular maintenance (combines updates + safety)
2. Use **deploy-rs** for quick configuration changes 2. Use **deploy-rs** for quick configuration changes
3. Keep **legacy deploy** as fallback method 3. Keep **legacy deploy** as fallback method

View file

@ -35,7 +35,6 @@
# User configuration # User configuration
../../modules/users/geir.nix ../../modules/users/geir.nix
../../modules/users/sma.nix
# Virtualization configuration # Virtualization configuration
../../modules/virtualization/incus.nix ../../modules/virtualization/incus.nix

View file

@ -12,7 +12,7 @@
"ll" = "eza -l --color=auto --group-directories-first"; "ll" = "eza -l --color=auto --group-directories-first";
"la" = "eza -la --color=auto --group-directories-first"; "la" = "eza -la --color=auto --group-directories-first";
"tree" = "eza --tree"; "tree" = "eza --tree";
# Safety first # Safety first
"rm" = "rm -i"; "rm" = "rm -i";
"mv" = "mv -i"; "mv" = "mv -i";

View file

@ -102,9 +102,9 @@ writeShellScriptBin "lab" ''
local dry_run="''${2:-false}" local dry_run="''${2:-false}"
log "Using deploy-rs for $machine deployment" log "Using deploy-rs for $machine deployment"
cd "$HOMELAB_ROOT" cd "$HOMELAB_ROOT"
if [[ "$dry_run" == "true" ]]; then if [[ "$dry_run" == "true" ]]; then
log "Running dry-run deployment..." log "Running dry-run deployment..."
if ! nix run github:serokell/deploy-rs -- ".#$machine" --dry-activate; then if ! nix run github:serokell/deploy-rs -- ".#$machine" --dry-activate; then
@ -126,20 +126,20 @@ writeShellScriptBin "lab" ''
update_flake() { update_flake() {
log "Updating flake inputs..." log "Updating flake inputs..."
cd "$HOMELAB_ROOT" cd "$HOMELAB_ROOT"
if ! nix flake update; then if ! nix flake update; then
error "Failed to update flake inputs" error "Failed to update flake inputs"
return 1 return 1
fi fi
log "Checking updated flake configuration..." log "Checking updated flake configuration..."
if ! nix flake check; then if ! nix flake check; then
error "Flake check failed after update" error "Flake check failed after update"
return 1 return 1
fi fi
success "Flake inputs updated successfully" success "Flake inputs updated successfully"
# Show what changed # Show what changed
log "Flake lock changes:" log "Flake lock changes:"
git diff --no-index /dev/null flake.lock | grep "+" | head -10 || true git diff --no-index /dev/null flake.lock | grep "+" | head -10 || true
@ -149,20 +149,20 @@ writeShellScriptBin "lab" ''
hybrid_update() { hybrid_update() {
local target="''${1:-all}" local target="''${1:-all}"
local dry_run="''${2:-false}" local dry_run="''${2:-false}"
log "Starting hybrid update process (target: $target, dry-run: $dry_run)" log "Starting hybrid update process (target: $target, dry-run: $dry_run)"
# Step 1: Update flake inputs # Step 1: Update flake inputs
if ! update_flake; then if ! update_flake; then
error "Failed to update flake - aborting hybrid update" error "Failed to update flake - aborting hybrid update"
return 1 return 1
fi fi
# Step 2: Deploy with deploy-rs # Step 2: Deploy with deploy-rs
if [[ "$target" == "all" ]]; then if [[ "$target" == "all" ]]; then
local machines=("sleeper-service" "grey-area" "reverse-proxy" "congenital-optimist") local machines=("sleeper-service" "grey-area" "reverse-proxy" "congenital-optimist")
local failed_machines=() local failed_machines=()
for machine in "''${machines[@]}"; do for machine in "''${machines[@]}"; do
log "Deploying updated configuration to $machine..." log "Deploying updated configuration to $machine..."
if deploy_rs_machine "$machine" "$dry_run"; then if deploy_rs_machine "$machine" "$dry_run"; then
@ -173,7 +173,7 @@ writeShellScriptBin "lab" ''
fi fi
echo "" echo ""
done done
if [[ ''${#failed_machines[@]} -eq 0 ]]; then if [[ ''${#failed_machines[@]} -eq 0 ]]; then
success "All machines updated successfully with hybrid approach!" success "All machines updated successfully with hybrid approach!"
else else
@ -216,31 +216,22 @@ writeShellScriptBin "lab" ''
show_status() { show_status() {
log "Home-lab infrastructure status:" log "Home-lab infrastructure status:"
# Check if -v (verbose) flag is passed for deploy-rs details
local verbose=0
local show_deploy_rs=0
for arg in "$@"; do
case "$arg" in
"-v"|"--verbose") verbose=1 ;;
"--deploy-rs") show_deploy_rs=1 ;;
"-vd"|"--verbose-deploy-rs") verbose=1; show_deploy_rs=1 ;;
esac
done
# Check congenital-optimist (local) # Check congenital-optimist (local)
if /run/current-system/sw/bin/systemctl is-active --quiet tailscaled; then if /run/current-system/sw/bin/systemctl is-active --quiet tailscaled; then
success " congenital-optimist: Online (local)" success " congenital-optimist: Online (local)"
if [[ $show_deploy_rs -eq 1 ]]; then
show_machine_deploy_info "congenital-optimist" "local"
fi
else else
warn " congenital-optimist: Tailscale inactive" warn " congenital-optimist: Tailscale inactive"
fi fi
# Check if -v (verbose) flag is passed
local verbose=0
if [[ "''${1:-}" == "-v" ]]; then
verbose=1
fi
# Check remote machines # Check remote machines
for machine in sleeper-service grey-area reverse-proxy; do for machine in sleeper-service grey-area reverse-proxy; do
local ssh_user="sma" # Using sma as the admin user for remote machines local ssh_user="sma" # Using sma as the admin user for remote machines
local connection_type=""
# Test SSH connectivity with debug info if in verbose mode # Test SSH connectivity with debug info if in verbose mode
if [[ $verbose -eq 1 ]]; then if [[ $verbose -eq 1 ]]; then
@ -262,10 +253,8 @@ writeShellScriptBin "lab" ''
# Use the specific Tailscale hostname for reverse-proxy # Use the specific Tailscale hostname for reverse-proxy
if ${openssh}/bin/ssh -o ConnectTimeout=5 -o BatchMode=yes "$ssh_user@reverse-proxy.tail807ea.ts.net" "echo OK" >/dev/null 2>&1; then if ${openssh}/bin/ssh -o ConnectTimeout=5 -o BatchMode=yes "$ssh_user@reverse-proxy.tail807ea.ts.net" "echo OK" >/dev/null 2>&1; then
success " $machine: Online (Tailscale)" success " $machine: Online (Tailscale)"
connection_type="reverse-proxy.tail807ea.ts.net"
elif ${openssh}/bin/ssh -o ConnectTimeout=2 -o BatchMode=yes "$ssh_user@$machine" "echo OK" >/dev/null 2>&1; then elif ${openssh}/bin/ssh -o ConnectTimeout=2 -o BatchMode=yes "$ssh_user@$machine" "echo OK" >/dev/null 2>&1; then
success " $machine: Online (LAN)" success " $machine: Online (LAN)"
connection_type="$machine"
else else
warn " $machine: Unreachable" warn " $machine: Unreachable"
if [[ $verbose -eq 1 ]]; then if [[ $verbose -eq 1 ]]; then
@ -277,70 +266,14 @@ writeShellScriptBin "lab" ''
else else
if ${openssh}/bin/ssh -o ConnectTimeout=2 -o BatchMode=yes "$ssh_user@$machine" "echo OK" >/dev/null 2>&1; then if ${openssh}/bin/ssh -o ConnectTimeout=2 -o BatchMode=yes "$ssh_user@$machine" "echo OK" >/dev/null 2>&1; then
success " $machine: Online (LAN)" success " $machine: Online (LAN)"
connection_type="$machine"
# Try with Tailscale hostname as fallback # Try with Tailscale hostname as fallback
elif ${openssh}/bin/ssh -o ConnectTimeout=3 -o BatchMode=yes "$ssh_user@$machine.tailnet" "echo OK" >/dev/null 2>&1; then elif ${openssh}/bin/ssh -o ConnectTimeout=3 -o BatchMode=yes "$ssh_user@$machine.tailnet" "echo OK" >/dev/null 2>&1; then
success " $machine: Online (Tailscale)" success " $machine: Online (Tailscale)"
connection_type="$machine.tailnet"
else else
warn " $machine: Unreachable" warn " $machine: Unreachable"
fi fi
fi fi
# Show deploy-rs information if requested and machine is reachable
if [[ $show_deploy_rs -eq 1 && -n "$connection_type" ]]; then
show_machine_deploy_info "$machine" "$connection_type"
fi
done done
if [[ $show_deploy_rs -eq 1 ]]; then
echo ""
log "💡 Use 'lab status --deploy-rs' to see deployment details"
log "💡 Use 'lab status -vd' for verbose deploy-rs information"
fi
}
# Show deploy-rs deployment information for a machine
show_machine_deploy_info() {
local machine="$1"
local connection="$2"
if [[ "$connection" == "local" ]]; then
# Local machine - get info directly
local current_gen=$(readlink /nix/var/nix/profiles/system | sed 's/.*system-\([0-9]*\)-link/\1/')
local system_closure=$(readlink -f /run/current-system)
local build_date=$(stat -c %y "$system_closure" 2>/dev/null | cut -d' ' -f1 2>/dev/null || echo "unknown")
echo " 📦 Generation: $current_gen"
echo " 📅 Build Date: $build_date"
echo " 📍 Store Path: $system_closure"
else
# Remote machine - get info via SSH
local ssh_user="sma"
local ssh_host="$connection"
local remote_info=$(${openssh}/bin/ssh -o ConnectTimeout=3 -o BatchMode=yes "$ssh_user@$ssh_host" "
current_gen=\$(readlink /nix/var/nix/profiles/system 2>/dev/null | sed 's/.*system-\([0-9]*\)-link/\1/' 2>/dev/null || echo 'unknown')
system_closure=\$(readlink -f /run/current-system 2>/dev/null || echo 'unknown')
build_date=\$(stat -c %y \$system_closure 2>/dev/null | cut -d' ' -f1 2>/dev/null || echo 'unknown')
uptime=\$(uptime -s 2>/dev/null || echo 'unknown')
echo \"gen:\$current_gen|path:\$system_closure|date:\$build_date|uptime:\$uptime\"
" 2>/dev/null)
if [[ -n "$remote_info" ]]; then
local gen=$(echo "$remote_info" | cut -d'|' -f1 | cut -d':' -f2)
local path=$(echo "$remote_info" | cut -d'|' -f2 | cut -d':' -f2)
local date=$(echo "$remote_info" | cut -d'|' -f3 | cut -d':' -f2)
local uptime=$(echo "$remote_info" | cut -d'|' -f4 | cut -d':' -f2)
echo " 📦 Generation: $gen"
echo " 📅 Build Date: $date"
echo " Boot Time: $uptime"
echo " 📍 Store Path: $(basename "$path")"
else
echo " Unable to retrieve deployment info"
fi
fi
} }
# Main command handling # Main command handling
@ -373,7 +306,7 @@ writeShellScriptBin "lab" ''
machine="$2" machine="$2"
dry_run="false" dry_run="false"
if [[ "''${3:-}" == "--dry-run" ]]; then if [[ "''${3:-}" == "--dry-run" ]]; then
dry_run="true" dry_run="true"
fi fi
@ -388,7 +321,7 @@ writeShellScriptBin "lab" ''
"hybrid-update") "hybrid-update")
target="''${2:-all}" target="''${2:-all}"
dry_run="false" dry_run="false"
if [[ "''${3:-}" == "--dry-run" ]]; then if [[ "''${3:-}" == "--dry-run" ]]; then
dry_run="true" dry_run="true"
fi fi
@ -397,8 +330,7 @@ writeShellScriptBin "lab" ''
;; ;;
"status") "status")
shift # Remove "status" from arguments show_status
show_status "$@" # Pass all remaining arguments to show_status
;; ;;
"update") "update")
@ -429,9 +361,7 @@ writeShellScriptBin "lab" ''
echo " hybrid-update [target] [opts] - Update flake + deploy with deploy-rs" echo " hybrid-update [target] [opts] - Update flake + deploy with deploy-rs"
echo " Target: machine name or 'all' (default)" echo " Target: machine name or 'all' (default)"
echo " Options: --dry-run" echo " Options: --dry-run"
echo " status [options] - Check infrastructure connectivity" echo " status - Check infrastructure connectivity"
echo " Options: -v (verbose), --deploy-rs (show deployment info)"
echo " -vd (verbose + deploy-rs info)"
echo "" echo ""
echo "Deployment Methods:" echo "Deployment Methods:"
echo " Legacy (SSH + rsync): Reliable, tested, slower" echo " Legacy (SSH + rsync): Reliable, tested, slower"
@ -459,8 +389,6 @@ writeShellScriptBin "lab" ''
echo "" echo ""
echo " # Status and monitoring" echo " # Status and monitoring"
echo " lab status # Check all machines" echo " lab status # Check all machines"
echo " lab status --deploy-rs # Show deployment details"
echo " lab status -vd # Verbose with deploy-rs info"
echo "" echo ""
echo " # Ollama AI tools" echo " # Ollama AI tools"
echo " ollama-cli status # Check Ollama service status" echo " ollama-cli status # Check Ollama service status"