Refactor: Simplify module structure and reorganize services
- Removed system/ directory, merged applications into users/geir.nix - Simplified fonts.nix to bare minimum (users can add more) - Moved transmission.nix to sleeper-service/services/ (machine-specific) - Organized grey-area services into services/ directory - Updated import paths and tested all configurations - Added research documentation for deploy-rs and GNU Stow
This commit is contained in:
Geir Okkenhaug Jerstad
parent
e976b14d19
commit
9837d82199
24 changed files with 832 additions and 959 deletions
236
research/deploy-rs.md
Normal file
236
research/deploy-rs.md
Normal file
|
|
@ -0,0 +1,236 @@
|
|||
# deploy-rs Research Summary
|
||||
|
||||
## Overview
|
||||
|
||||
**deploy-rs** is a Rust-based deployment tool specifically designed for NixOS flakes. It provides declarative, reliable, and efficient deployment of NixOS configurations to remote machines with advanced features like rollback capabilities, health checks, and parallel deployments.
|
||||
|
||||
**Repository**: https://github.com/serokell/deploy-rs
|
||||
**Status**: Actively maintained by Serokell
|
||||
**Language**: Rust (fast, reliable, memory-safe)
|
||||
|
||||
## Key Features
|
||||
|
||||
### 🚀 **Core Capabilities**
|
||||
- **Flake-native**: Built specifically for Nix flakes (no legacy nix-env/channels)
|
||||
- **Multi-target deployment**: Deploy to multiple machines simultaneously
|
||||
- **Automatic rollback**: Failed deployments automatically revert to previous generation
|
||||
- **Health checks**: Configurable post-deployment validation
|
||||
- **SSH-based**: Uses SSH for secure remote deployment (like our current lab tool)
|
||||
- **Profile management**: Supports system, user, and custom profiles
|
||||
|
||||
### 🔧 **Advanced Features**
|
||||
- **Parallel deployment**: Deploy to multiple machines concurrently
|
||||
- **Interactive confirmation**: Can prompt before applying changes
|
||||
- **Dry-run mode**: Preview changes without applying them
|
||||
- **Magic rollback**: Automatic rollback on deployment failures or health check failures
|
||||
- **Custom activation**: Define custom activation scripts and checks
|
||||
- **Sudo handling**: Intelligent sudo privilege escalation
|
||||
|
||||
### 📊 **Reliability Features**
|
||||
- **Atomic deployments**: Either succeeds completely or rolls back
|
||||
- **Connection resilience**: Handles SSH connection issues gracefully
|
||||
- **Generation tracking**: Keeps track of deployment history
|
||||
- **Activation timeout**: Prevents hanging deployments
|
||||
- **Health check timeout**: Configurable validation windows
|
||||
|
||||
## Configuration Structure
|
||||
|
||||
Deploy-rs uses a declarative configuration format in your flake:
|
||||
|
||||
```nix
|
||||
# flake.nix
|
||||
{
|
||||
# ... existing flake configuration
|
||||
|
||||
deploy.nodes = {
|
||||
sleeper-service = {
|
||||
hostname = "sleeper-service.tail807ea.ts.net";
|
||||
profiles.system = {
|
||||
user = "root";
|
||||
path = deploy-rs.lib.x86_64-linux.activate.nixos
|
||||
self.nixosConfigurations.sleeper-service;
|
||||
sshUser = "sma";
|
||||
sudo = "sudo -u";
|
||||
};
|
||||
};
|
||||
|
||||
grey-area = {
|
||||
hostname = "grey-area.tail807ea.ts.net";
|
||||
profiles.system = {
|
||||
user = "root";
|
||||
path = deploy-rs.lib.x86_64-linux.activate.nixos
|
||||
self.nixosConfigurations.grey-area;
|
||||
sshUser = "sma";
|
||||
sudo = "sudo -u";
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
# Health checks
|
||||
deploy.nodes.sleeper-service.profiles.system.activationTimeout = 240;
|
||||
deploy.nodes.sleeper-service.profiles.system.confirmTimeout = 30;
|
||||
}
|
||||
```
|
||||
|
||||
## Command Examples
|
||||
|
||||
```bash
|
||||
# Deploy to single machine
|
||||
deploy '.#sleeper-service'
|
||||
|
||||
# Deploy to all machines
|
||||
deploy '.#'
|
||||
|
||||
# Dry run (check what would be deployed)
|
||||
deploy '.#sleeper-service' -- --dry-activate
|
||||
|
||||
# Skip health checks
|
||||
deploy '.#sleeper-service' -- --skip-checks
|
||||
|
||||
# Interactive confirmation
|
||||
deploy '.#sleeper-service' -- --confirm-timeout 60
|
||||
|
||||
# Deploy specific profile
|
||||
deploy '.#sleeper-service.system'
|
||||
```
|
||||
|
||||
## Comparison with Current `lab` Tool
|
||||
|
||||
| Feature | Current `lab` Tool | deploy-rs |
|
||||
|---------|-------------------|-----------|
|
||||
| **Language** | Shell script | Rust (compiled) |
|
||||
| **Performance** | Good | Excellent |
|
||||
| **Parallel deployment** | ❌ | ✅ |
|
||||
| **Automatic rollback** | ❌ | ✅ |
|
||||
| **Health checks** | ❌ | ✅ |
|
||||
| **Flake-native** | ✅ | ✅ |
|
||||
| **SSH-based** | ✅ | ✅ |
|
||||
| **Status monitoring** | ✅ | Limited |
|
||||
| **Custom workflows** | ✅ | Limited |
|
||||
| **Learning curve** | Low | Medium |
|
||||
| **Configuration** | Shell script | Nix flake |
|
||||
|
||||
## Advantages of deploy-rs
|
||||
|
||||
### ✅ **Production-Ready Features**
|
||||
- **Reliability**: Automatic rollback prevents broken deployments
|
||||
- **Speed**: Rust performance + parallel deployment
|
||||
- **Safety**: Health checks ensure successful activation
|
||||
- **Consistency**: Declarative configuration in flake
|
||||
|
||||
### ✅ **Operational Benefits**
|
||||
- **Reduced downtime**: Atomic deployments with quick rollback
|
||||
- **Error handling**: Sophisticated error recovery mechanisms
|
||||
- **Audit trail**: Built-in deployment history tracking
|
||||
- **Validation**: Pre and post-deployment checks
|
||||
|
||||
### ✅ **Scale Benefits**
|
||||
- **Multi-machine**: Deploy entire infrastructure simultaneously
|
||||
- **Efficiency**: Parallel operations reduce total deployment time
|
||||
- **Resource management**: Better handling of resource conflicts
|
||||
|
||||
## Disadvantages & Limitations
|
||||
|
||||
### ❌ **Current Limitations**
|
||||
- **Status monitoring**: No equivalent to `lab status` for infrastructure overview
|
||||
- **Custom workflows**: Less flexible than shell scripts for custom operations
|
||||
- **Learning curve**: Requires understanding deploy-rs configuration syntax
|
||||
- **Debugging**: Rust binary vs readable shell script
|
||||
- **Community size**: Smaller ecosystem compared to traditional tools
|
||||
|
||||
### ❌ **Home Lab Specific Concerns**
|
||||
- **Overkill factor**: May be complex for 3-4 machine home lab
|
||||
- **Customization**: Our `lab` tool has home lab specific features
|
||||
- **Integration**: Would need to replicate status monitoring capabilities
|
||||
- **Development workflow**: Less hackable than shell scripts
|
||||
|
||||
## Implementation Recommendations
|
||||
|
||||
### 🎯 **Hybrid Approach (Recommended)**
|
||||
Keep the best of both tools:
|
||||
|
||||
1. **Use deploy-rs for deployments**:
|
||||
```bash
|
||||
lab deploy-rs sleeper-service # Use deploy-rs backend
|
||||
lab deploy grey-area # Current shell script method
|
||||
```
|
||||
|
||||
2. **Keep `lab` tool for status and management**:
|
||||
```bash
|
||||
lab status # Infrastructure overview
|
||||
lab check sleeper-service # Health monitoring
|
||||
lab logs grey-area # Log access
|
||||
```
|
||||
|
||||
### 🔧 **Migration Strategy**
|
||||
|
||||
**Phase 1: Evaluation** (Current)
|
||||
- Add deploy-rs configuration to flake
|
||||
- Test deployment on non-critical machine
|
||||
- Compare reliability and performance
|
||||
|
||||
**Phase 2: Gradual Adoption**
|
||||
- Migrate stable machines to deploy-rs
|
||||
- Keep custom `lab` commands for monitoring
|
||||
- Maintain shell script fallback
|
||||
|
||||
**Phase 3: Integration**
|
||||
- Enhance `lab` tool to use deploy-rs as backend
|
||||
- Add deploy-rs specific features to `lab status`
|
||||
- Maintain unified interface
|
||||
|
||||
### 📝 **Flake Integration Example**
|
||||
|
||||
```nix
|
||||
# Add to flake.nix inputs
|
||||
inputs.deploy-rs.url = "github:serokell/deploy-rs";
|
||||
|
||||
# Add to flake outputs
|
||||
deploy = {
|
||||
nodes = {
|
||||
sleeper-service = {
|
||||
hostname = "10.0.0.8"; # Or Tailscale hostname
|
||||
profiles.system = {
|
||||
user = "root";
|
||||
path = deploy-rs.lib.x86_64-linux.activate.nixos
|
||||
self.nixosConfigurations.sleeper-service;
|
||||
sshUser = "sma";
|
||||
sshOpts = ["-p" "22"];
|
||||
fastConnection = false; # For home network
|
||||
autoRollback = true;
|
||||
magicRollback = true;
|
||||
activationTimeout = 180;
|
||||
confirmTimeout = 30;
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
# Health checks can reference systemd services
|
||||
deploy.nodes.sleeper-service.profiles.system.activationTimeout = 240;
|
||||
```
|
||||
|
||||
## Conclusion & Recommendation
|
||||
|
||||
### 🎯 **For Our Home Lab**
|
||||
|
||||
**deploy-rs is valuable for:**
|
||||
- Production-quality deployments with rollback safety
|
||||
- Parallel deployment when infrastructure grows
|
||||
- Reduced risk of broken remote systems
|
||||
- Professional deployment practices
|
||||
|
||||
**Our current `lab` tool excels at:**
|
||||
- Home lab specific status monitoring
|
||||
- Custom workflows and debugging
|
||||
- Simple, hackable shell script approach
|
||||
- Tailored for our specific infrastructure
|
||||
|
||||
### 📋 **Action Plan**
|
||||
|
||||
1. **Immediate**: Add deploy-rs configuration to flake (low effort, high learning)
|
||||
2. **Short-term**: Test deploy-rs on sleeper-service alongside current method
|
||||
3. **Medium-term**: Consider hybrid approach - deploy-rs for deployment, `lab` for monitoring
|
||||
4. **Long-term**: Evaluate full migration based on home lab growth and complexity needs
|
||||
|
||||
**Verdict**: deploy-rs is a professional-grade tool that would enhance our deployment reliability. The hybrid approach allows us to benefit from deploy-rs's deployment safety while keeping our custom infrastructure monitoring capabilities.
|
||||
300
research/gnu-stow.md
Normal file
300
research/gnu-stow.md
Normal file
|
|
@ -0,0 +1,300 @@
|
|||
# GNU Stow Research Summary
|
||||
|
||||
## Overview
|
||||
|
||||
**GNU Stow** is a symlink farm manager that helps organize and deploy dotfiles and software packages. It creates symbolic links from a source directory to a target directory, making it ideal for managing dotfiles across multiple systems without copying files.
|
||||
|
||||
**Repository**: https://www.gnu.org/software/stow/
|
||||
**Language**: Perl
|
||||
**License**: GPL-3.0+
|
||||
**Status**: Mature, stable, widely adopted
|
||||
|
||||
## Core Concept
|
||||
|
||||
Stow works by creating symbolic links from a "stow directory" (source) to a "target directory" (typically `$HOME`). Each subdirectory in the stow directory represents a "package" that can be independently stowed or unstowed.
|
||||
|
||||
```
|
||||
dotfiles/ # Stow directory
|
||||
├── zsh/ # Package: zsh configuration
|
||||
│ └── .zshrc # Will link to ~/.zshrc
|
||||
├── emacs/ # Package: emacs configuration
|
||||
│ └── .emacs.d/ # Will link to ~/.emacs.d/
|
||||
│ └── init.el
|
||||
└── git/ # Package: git configuration
|
||||
└── .gitconfig # Will link to ~/.gitconfig
|
||||
```
|
||||
|
||||
## Key Features
|
||||
|
||||
### 🔗 **Symlink Management**
|
||||
- **Non-destructive**: Creates symlinks without overwriting existing files
|
||||
- **Conflict detection**: Warns about file conflicts before creating links
|
||||
- **Tree folding**: Optimizes symlink structure for efficiency
|
||||
- **Partial deployment**: Deploy only specific packages (e.g., just zsh config)
|
||||
|
||||
### 📦 **Package-Based Organization**
|
||||
- **Modular structure**: Each application gets its own package directory
|
||||
- **Selective deployment**: Install only needed configurations per machine
|
||||
- **Easy maintenance**: Add/remove configurations without affecting others
|
||||
- **Version control friendly**: Git can track each package independently
|
||||
|
||||
### 🛡️ **Safety Features**
|
||||
- **Dry-run mode**: Preview what would be done without making changes
|
||||
- **Conflict resolution**: Handles existing files and directories gracefully
|
||||
- **Rollback capability**: Easy to unstow (remove) configurations
|
||||
- **Target verification**: Ensures target directory exists and is writable
|
||||
|
||||
## Command Examples
|
||||
|
||||
```bash
|
||||
# Basic stow operations
|
||||
cd ~/dotfiles
|
||||
stow zsh # Deploy zsh configuration
|
||||
stow emacs git # Deploy multiple packages
|
||||
stow --target=/opt/local mypackage # Custom target directory
|
||||
|
||||
# Management operations
|
||||
stow --delete zsh # Remove zsh symlinks (unstow)
|
||||
stow --restow emacs # Restow (unstow then stow again)
|
||||
stow --simulate zsh # Dry run - show what would happen
|
||||
|
||||
# Advanced usage
|
||||
stow --verbose zsh # Verbose output
|
||||
stow --no-folding emacs # Disable tree folding optimization
|
||||
stow --ignore='\.DS_Store' zsh # Ignore certain files
|
||||
```
|
||||
|
||||
## Directory Structure Example
|
||||
|
||||
```
|
||||
~/dotfiles/ # Stow directory
|
||||
├── zsh/ # Package: ZSH
|
||||
│ ├── .zshrc # → ~/.zshrc
|
||||
│ └── .config/ # → ~/.config/
|
||||
│ └── zsh/ # → ~/.config/zsh/
|
||||
│ └── aliases.zsh # → ~/.config/zsh/aliases.zsh
|
||||
├── emacs/ # Package: Emacs
|
||||
│ ├── .emacs.d/ # → ~/.emacs.d/
|
||||
│ │ ├── init.el # → ~/.emacs.d/init.el
|
||||
│ │ └── config.org # → ~/.emacs.d/config.org
|
||||
│ └── .config/ # → ~/.config/
|
||||
│ └── emacs/ # → ~/.config/emacs/
|
||||
├── git/ # Package: Git
|
||||
│ ├── .gitconfig # → ~/.gitconfig
|
||||
│ └── .gitignore_global # → ~/.gitignore_global
|
||||
└── nixos/ # Package: NixOS-specific
|
||||
└── .config/ # → ~/.config/
|
||||
└── nixos/ # → ~/.config/nixos/
|
||||
└── user.nix # → ~/.config/nixos/user.nix
|
||||
```
|
||||
|
||||
## Comparison with Current Org-Mode Approach
|
||||
|
||||
| Feature | Current Org-Mode | GNU Stow |
|
||||
|---------|------------------|----------|
|
||||
| **Configuration format** | Literate programming | Traditional dotfiles |
|
||||
| **Deployment method** | Tangle + copy/symlink | Automatic symlinking |
|
||||
| **Documentation** | Embedded in code | Separate documentation |
|
||||
| **Version control** | Single org file | Multiple files in packages |
|
||||
| **Learning curve** | Steep (Emacs/Org) | Gentle (simple concept) |
|
||||
| **Flexibility** | Very high | Moderate |
|
||||
| **Maintenance** | Manual tangling | Automatic linking |
|
||||
| **Cross-platform** | Excellent | Excellent |
|
||||
| **NixOS integration** | Native | External tool |
|
||||
|
||||
## Advantages of GNU Stow
|
||||
|
||||
### ✅ **Simplicity & Reliability**
|
||||
- **Minimal learning curve**: Understand symlinks = understand Stow
|
||||
- **No dependencies**: Works on any Unix-like system with Perl
|
||||
- **Predictable behavior**: Simple, well-defined symlink creation rules
|
||||
- **Mature and stable**: Decades of development and testing
|
||||
|
||||
### ✅ **Flexible Organization**
|
||||
- **Package-based**: Logical separation of application configurations
|
||||
- **Selective deployment**: Deploy only needed configs per machine
|
||||
- **Easy experimentation**: Test new configs without affecting others
|
||||
- **Conflict handling**: Safe deployment with conflict detection
|
||||
|
||||
### ✅ **Version Control Benefits**
|
||||
- **Git-friendly**: Each package can be tracked separately
|
||||
- **Branching**: Different branches for different machine types
|
||||
- **History**: Track changes to individual application configs
|
||||
- **Collaboration**: Easy to share specific application configurations
|
||||
|
||||
### ✅ **Multi-Machine Management**
|
||||
```bash
|
||||
# Different configs for different machine types
|
||||
stow --target=/home/geir desktop # Desktop-specific configs
|
||||
stow --target=/home/sma server # Server-specific configs
|
||||
|
||||
# Machine-specific packages
|
||||
dotfiles/
|
||||
├── desktop-geir/ # Only for desktop user geir
|
||||
├── server-sma/ # Only for server user sma
|
||||
└── common/ # Shared configurations
|
||||
```
|
||||
|
||||
## Disadvantages & Limitations
|
||||
|
||||
### ❌ **Compared to Org-Mode Approach**
|
||||
- **No literate programming**: Configuration and documentation separate
|
||||
- **Less integration**: Not as tightly integrated with Emacs workflow
|
||||
- **File-based**: No ability to generate configs from templates
|
||||
- **Static configuration**: Less dynamic than org-mode tangling
|
||||
|
||||
### ❌ **General Limitations**
|
||||
- **Symlink visibility**: Some applications don't handle symlinks well
|
||||
- **Permission issues**: Target directory permissions must be correct
|
||||
- **Perl dependency**: Requires Perl runtime (usually not an issue)
|
||||
- **No config generation**: Can't generate configs based on system state
|
||||
|
||||
## Integration with Current NixOS Setup
|
||||
|
||||
### 🎯 **Hybrid Approach Options**
|
||||
|
||||
**Option 1: Stow for Traditional Dotfiles**
|
||||
```bash
|
||||
# Keep org-mode for complex Emacs config
|
||||
# Use Stow for simple dotfiles
|
||||
dotfiles/
|
||||
├── emacs.org # Keep current literate approach
|
||||
├── zsh/ # Simple dotfiles via Stow
|
||||
│ └── .zshrc
|
||||
└── git/
|
||||
└── .gitconfig
|
||||
```
|
||||
|
||||
**Option 2: User-Specific Deployment**
|
||||
```bash
|
||||
# Different approaches per user
|
||||
~/dotfiles-geir/ # Org-mode for desktop user
|
||||
~/dotfiles-sma/ # Stow for server user (simpler configs)
|
||||
```
|
||||
|
||||
**Option 3: NixOS + Stow Integration**
|
||||
```nix
|
||||
# In NixOS configuration
|
||||
environment.systemPackages = [ pkgs.stow ];
|
||||
|
||||
# User activation script
|
||||
system.userActivationScripts.stow-dotfiles = ''
|
||||
cd /home/geir/dotfiles
|
||||
${pkgs.stow}/bin/stow --target=/home/geir common zsh git
|
||||
'';
|
||||
```
|
||||
|
||||
### 🔧 **Implementation Strategies**
|
||||
|
||||
**For Server Configurations (sma user):**
|
||||
- Simple, minimal dotfiles via Stow
|
||||
- Basic shell, git, and utility configurations
|
||||
- No complex literate programming needed
|
||||
|
||||
**For Desktop Configurations (geir user):**
|
||||
- Keep org-mode for Emacs (complex, well-integrated)
|
||||
- Consider Stow for simpler applications (git, zsh aliases)
|
||||
- Hybrid approach based on complexity
|
||||
|
||||
## Practical Examples
|
||||
|
||||
### 📝 **Basic Setup**
|
||||
```bash
|
||||
# Initialize dotfiles repository
|
||||
mkdir ~/dotfiles
|
||||
cd ~/dotfiles
|
||||
|
||||
# Create package structure
|
||||
mkdir -p zsh git emacs
|
||||
|
||||
# Add configurations
|
||||
echo 'alias ll="ls -la"' > zsh/.zshrc
|
||||
echo '[user]
|
||||
name = Geir Okkenhaug Jerstad
|
||||
email = geir@geokkjer.eu' > git/.gitconfig
|
||||
|
||||
# Deploy configurations
|
||||
stow zsh git
|
||||
```
|
||||
|
||||
### 🏠 **Home Lab Specific Usage**
|
||||
```bash
|
||||
# Machine-specific dotfiles
|
||||
dotfiles/
|
||||
├── server-common/ # Shared server configs
|
||||
│ ├── .zshrc
|
||||
│ └── .vimrc
|
||||
├── sleeper-service/ # NFS-specific configs
|
||||
│ └── .config/
|
||||
│ └── nfs/
|
||||
└── grey-area/ # Git server specific
|
||||
└── .gitconfig
|
||||
|
||||
# Deploy on sleeper-service
|
||||
stow server-common sleeper-service
|
||||
|
||||
# Deploy on grey-area
|
||||
stow server-common grey-area
|
||||
```
|
||||
|
||||
### 🔄 **Migration Strategy**
|
||||
```bash
|
||||
# Phase 1: Extract simple configs from org-mode
|
||||
emacs config.org
|
||||
# Tangle simple configs to separate files
|
||||
# Create Stow packages for extracted configs
|
||||
|
||||
# Phase 2: Test Stow deployment
|
||||
stow --simulate zsh # Test before applying
|
||||
stow zsh # Deploy if tests pass
|
||||
|
||||
# Phase 3: Maintain hybrid approach
|
||||
# Keep org-mode for complex configs (Emacs)
|
||||
# Use Stow for simple configs (shell, git)
|
||||
```
|
||||
|
||||
## Integration with Current Workflow
|
||||
|
||||
### 🎯 **Recommended Approach for Home Lab**
|
||||
|
||||
**Keep Current Org-Mode For:**
|
||||
- Emacs configuration (complex, well-integrated)
|
||||
- Complex, documented configurations
|
||||
- Configurations that benefit from literate programming
|
||||
|
||||
**Use GNU Stow For:**
|
||||
- Server user (sma) configurations
|
||||
- Simple shell configurations
|
||||
- Git configurations
|
||||
- Application-specific dotfiles that don't need documentation
|
||||
|
||||
**Implementation Plan:**
|
||||
1. **Create stow repository** for server configs
|
||||
2. **Test on sleeper-service** with sma user
|
||||
3. **Gradually migrate** simple configs from org-mode
|
||||
4. **Maintain hybrid approach** based on complexity needs
|
||||
|
||||
## Conclusion & Recommendation
|
||||
|
||||
### 🎯 **For Our Home Lab Context**
|
||||
|
||||
**GNU Stow is excellent for:**
|
||||
- Server configurations (sma user)
|
||||
- Simple, stable dotfiles
|
||||
- Multi-machine deployments
|
||||
- Configurations that don't need complex documentation
|
||||
|
||||
**Current org-mode approach excels for:**
|
||||
- Complex Emacs configurations
|
||||
- Documented, literate configurations
|
||||
- Generated configurations
|
||||
- Desktop user (geir) complex setups
|
||||
|
||||
### 📋 **Action Plan**
|
||||
|
||||
1. **Immediate**: Create stow-based dotfiles repository for server users
|
||||
2. **Short-term**: Deploy simple sma user configs via Stow on sleeper-service
|
||||
3. **Medium-term**: Evaluate migration of simple configs from org-mode
|
||||
4. **Long-term**: Maintain hybrid approach - Stow for simple, org-mode for complex
|
||||
|
||||
**Verdict**: GNU Stow is an excellent complement to our current org-mode approach. Use Stow for simple, server-side configurations while keeping org-mode for complex desktop configurations. This provides the best of both worlds - simplicity where appropriate, power where needed.
|
||||
Loading…
Add table
Add a link
Reference in a new issue