Compare commits

...

7 commits

Author SHA1 Message Date
Geir Okkenhaug Jerstad
fed1c5a1f8 docs: update templates and documentation to professional tone
- Add git commit message template with comprehensive guidelines
- Update PR template to remove emojis and casual language
- Rewrite README.md with professional, technical approach
- Update BRANCHING_STRATEGY.md to match new tone
- Backup original README as README_old.md

Templates now align with infrastructure documentation standards
and provide clear guidance for contributions.
2025-06-07 17:39:39 +00:00
Geir Okkenhaug Jerstad
7aafd4cdd8 docs: Add Forgejo Git configuration instructions
- Correct SSH URL format: forgejo@git.geokkjer.eu:1337
- Document SSH port forwarding setup (1337 -> grey-area:22)
- Add remote configuration examples
- Clarify primary vs backup repository roles
2025-06-07 16:46:09 +00:00
Geir Okkenhaug Jerstad
c8bee48ee3 Update plan.md: GNU Stow + literate Emacs approach, deploy-rs migration planning
- Phase 4: Restructured to use GNU Stow for regular dotfiles + literate programming for Emacs only
- Added comprehensive package structure for Stow deployment
- Elevated deploy-rs migration to high priority with detailed configuration examples
- Updated status to reflect 4/4 machines fully operational with complete service stack
- Added recent critical issue resolution documentation
- Updated next phase priorities to reflect new dotfiles approach
2025-06-07 16:44:11 +00:00
Geir Okkenhaug Jerstad
4a57978f45 fixed nfs 2025-06-07 16:33:34 +00:00
Geir Okkenhaug Jerstad
9bfddf14ce treying to get nfs to work 2025-06-07 15:29:28 +00:00
Geir Okkenhaug Jerstad
2d3728f28b feat: create shared extraHosts module with Tailscale IPs
- Create modules/network/extraHosts.nix with Tailscale IP mappings
- Replace hardcoded networking.extraHosts in all machine configs
- Add extraHosts module import to all machines
- Enable Tailscale service by default in the module
- Use Tailscale mesh network IPs for reliable connectivity
2025-06-07 15:07:17 +00:00
Geir Okkenhaug Jerstad
fa2b84cf65 fix: resolve sma user definition conflict between modules
- Remove duplicate sma user definition from incus.nix module
- The sma user is properly defined in modules/users/sma.nix with incus-admin group
- This resolves the isNormalUser/isSystemUser assertion failure blocking congenital-optimist rebuild
- Clean up grey-area configuration and modularize services
- Update SSH keys with correct IP addresses for grey-area and reverse-proxy
2025-06-07 16:58:22 +02:00
33 changed files with 1493 additions and 657 deletions

View file

@ -1,118 +0,0 @@
## 🏠 Home Lab Configuration Change
### 📋 Description
<!-- Describe what this PR does and why -->
### 🎯 Type of Change
<!-- Mark all that apply -->
- [ ] 🐛 Bug fix (non-breaking change that fixes an issue)
- [ ] ✨ New feature (non-breaking change that adds functionality)
- [ ] 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] 📚 Documentation update
- [ ] 🔧 Configuration change
- [ ] 🏗️ Infrastructure change
- [ ] 🔒 Security update
### 🖥️ Affected Machines
<!-- Mark all machines affected by this change -->
- [ ] `congenital-optimist` (AMD workstation)
- [ ] `sleeper-service` (Intel file server)
- [ ] Both machines
- [ ] New machine configuration
### 🧪 Testing Performed
<!-- Describe how you tested these changes -->
- [ ] `nix flake check` passes
- [ ] `nixos-rebuild test --flake` successful
- [ ] `nixos-rebuild build --flake` successful
- [ ] Manual testing of affected functionality
- [ ] Rollback tested (if applicable)
### 📝 Testing Checklist
<!-- Check all items that were verified -->
#### System Functionality
- [ ] System boots successfully
- [ ] Network connectivity works
- [ ] Services start correctly
- [ ] No error messages in logs
#### Desktop Environment (if applicable)
- [ ] Desktop environment launches
- [ ] Applications start correctly
- [ ] Hardware acceleration works
- [ ] Audio/video functional
#### Virtualization (if applicable)
- [ ] Incus containers work
- [ ] Libvirt VMs functional
- [ ] Podman containers operational
- [ ] Network isolation correct
#### Development Environment (if applicable)
- [ ] Editors launch correctly
- [ ] Language servers work
- [ ] Build tools functional
- [ ] Git configuration correct
#### File Services (if applicable)
- [ ] NFS mounts accessible
- [ ] Samba shares working
- [ ] Backup services operational
- [ ] Storage pools healthy
### 🔒 Security Considerations
<!-- Any security implications of this change -->
- [ ] No new attack vectors introduced
- [ ] Secrets properly managed
- [ ] Firewall rules reviewed
- [ ] User permissions appropriate
### 📖 Documentation
<!-- Documentation changes -->
- [ ] README.md updated (if needed)
- [ ] Module documentation updated
- [ ] plan.md updated (if needed)
- [ ] Comments added to complex configurations
### 🔄 Rollback Plan
<!-- How to rollback if something goes wrong -->
- [ ] Previous configuration saved
- [ ] ZFS snapshot created
- [ ] Rollback procedure documented
- [ ] Emergency access method available
### 📋 Deployment Notes
<!-- Special considerations for deployment -->
- [ ] No special deployment steps required
- [ ] Requires manual intervention: <!-- describe -->
- [ ] Needs coordination with other changes
- [ ] Breaking change requires communication
### 🔗 Related Issues
<!-- Link any related issues -->
Fixes #<!-- issue number -->
Related to #<!-- issue number -->
### 📸 Screenshots/Logs
<!-- Add any relevant screenshots or log outputs -->
### ✅ Final Checklist
<!-- Verify before submitting -->
- [ ] I have tested this change locally
- [ ] I have updated documentation as needed
- [ ] I have considered the impact on other machines
- [ ] I have verified the rollback plan
- [ ] I have checked for any secrets in the code
- [ ] This change follows the repository's coding standards
### 🧠 Additional Context
<!-- Add any other context about the PR here -->
---
**Reviewer Guidelines:**
1. Verify all testing checkboxes are complete
2. Review configuration changes for security implications
3. Ensure rollback plan is realistic
4. Check that documentation is updated
5. Validate CI pipeline passes

56
.gitmessage Normal file
View file

@ -0,0 +1,56 @@
# <type>(<scope>): <description>
#
# <body>
#
# <footer>
# --- COMMIT MESSAGE GUIDELINES ---
#
# FORMAT:
# <type>(<scope>): <description>
#
# [optional body]
#
# [optional footer]
#
# TYPES:
# feat - New feature or module
# fix - Bug fix
# docs - Documentation changes
# style - Formatting, missing semicolons, etc.
# refactor - Code refactoring
# test - Adding tests
# chore - Maintenance tasks
#
# SCOPES:
# Machine: (congenital-optimist), (sleeper-service), (grey-area), (reverse-proxy)
# Module: (desktop), (virtualization), (users), (network), (security)
# Config: (flake), (ci), (git)
# Docs: (readme), (plan), (branching)
#
# DESCRIPTION:
# - Use imperative mood: "add", "fix", "update" (not "added", "fixed", "updated")
# - Keep under 50 characters
# - Start with lowercase
# - No period at end
#
# BODY:
# - Explain what and why, not how
# - Wrap at 72 characters
# - Separate from description with blank line
#
# FOOTER:
# - Reference issues: "Fixes #123", "Related to #456"
# - Breaking changes: "BREAKING CHANGE: description"
#
# EXAMPLES:
# feat(desktop): add cosmic desktop environment module
# fix(virtualization): resolve incus networking configuration
# docs(readme): update installation instructions
# refactor(modules): reorganize desktop environment structure
# chore(ci): update github actions workflow
# feat(sleeper-service): implement nfs server configuration
# fix(congenital-optimist): resolve zfs boot failure
#
# Keep commits focused and atomic. Each commit should represent
# a single logical change that can be easily reviewed and reverted.

View file

@ -1,8 +1,8 @@
# 🌳 Git Branching Strategy for Home Lab Infrastructure
# Git Branching Strategy for Infrastructure Management
## Branch Structure
### 🚀 Main Branches
### Main Branches
#### `main`
- **Purpose**: Production-ready configurations
@ -16,7 +16,7 @@
- **Merging**: Features merge here first
- **Deployment**: Deployed to staging/test environments
### 🔧 Supporting Branches
### Supporting Branches
#### Feature Branches: `feature/<description>`
- **Purpose**: Development of new features or modules
@ -43,7 +43,7 @@
- **Scope**: Single module focus
- **Testing**: Module-specific testing
### 🏷️ Tagging Strategy
### Tagging Strategy
#### Version Tags: `v<major>.<minor>.<patch>`
- **Purpose**: Mark stable releases
@ -62,7 +62,7 @@
- **Format**: `phase-1-complete`, `phase-2-complete`
- **Documentation**: Link to plan.md milestones
## 🔄 Workflow Examples
## Workflow Examples
### Standard Feature Development
```bash
@ -122,7 +122,7 @@ git checkout develop
git merge hotfix/zfs-boot-failure
```
## 📋 Commit Convention
## Commit Convention
### Format
```
@ -157,7 +157,7 @@ refactor(modules): reorganize desktop environment modules
chore(ci): update GitHub Actions workflow
```
## 🛡️ Branch Protection Rules
## Branch Protection Rules
### Main Branch Protection
- **Required Reviews**: 1 reviewer minimum
@ -173,7 +173,7 @@ chore(ci): update GitHub Actions workflow
- **Auto-merge**: Allow auto-merge after checks
- **Force Push**: Disabled for others
## 🔄 Merge Strategies
## Merge Strategies
### Feature to Develop
- **Strategy**: Squash and merge
@ -190,7 +190,7 @@ chore(ci): update GitHub Actions workflow
- **Reason**: Immediate deployment needed
- **Testing**: Minimal but critical testing
## 🚀 Deployment Strategy
## Deployment Strategy
### Automatic Deployment
- **main** → Production machines (congenital-optimist, sleeper-service)
@ -213,7 +213,7 @@ git checkout v1.0.0
sudo nixos-rebuild switch --flake .#congenital-optimist
```
## 📊 Branch Lifecycle
## Branch Lifecycle
### Weekly Maintenance
- **Monday**: Review open feature branches
@ -227,7 +227,7 @@ sudo nixos-rebuild switch --flake .#congenital-optimist
- Update documentation
- Security audit of configurations
## 🎯 Best Practices
## Best Practices
### Branch Naming
- Use descriptive names: `feature/improve-zfs-performance`

336
README.md
View file

@ -1,257 +1,217 @@
# 🏠 NixOS Home Lab Adventures
# NixOS Home Lab Infrastructure
[![NixOS](https://img.shields.io/badge/NixOS-25.05-blue.svg)](https://nixos.org/)
[![Flakes](https://img.shields.io/badge/Nix-Flakes-green.svg)](https://nixos.wiki/wiki/Flakes)
[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
A personal journey into NixOS flakes and home lab tinkering. This is my playground for learning declarative system configuration and building a multi-machine setup that's both fun and functional.
Modular NixOS flake configuration for multi-machine home lab infrastructure. Features declarative system configuration, centralized user management, and scalable service deployment across development workstations and server infrastructure.
## 🚀 Getting Started
# Vibe DevSecOpsing with claud-sonnet 4 and github-copilot
Want to try this out? Here's how to get rolling:
## Quick Start
```bash
# Grab the repo
# Clone repository
git clone <repository-url> Home-lab
cd Home-lab
# Make sure everything looks good
# Validate configuration
nix flake check
# Test it out (won't mess with your current setup)
sudo nixos-rebuild test --flake .#congenital-optimist
# Test configuration (temporary, reverts on reboot)
sudo nixos-rebuild test --flake .#<machine-name>
# If you're happy with it, make it permanent
sudo nixos-rebuild switch --flake .#congenital-optimist
# Apply configuration permanently
sudo nixos-rebuild switch --flake .#<machine-name>
```
## 🏗️ What We're Working With
## Architecture Overview
### The Machines
- **`congenital-optimist`** - My main AMD Threadripper beast for development and experimentation
- **`sleeper-service`** - Intel Xeon E3-1230 V2 running file server duties (the quiet workhorse)
### Machine Types
- **Development Workstation** - High-performance development environment with desktop environments
- **File Server** - ZFS storage with NFS services and media management
- **Application Server** - Containerized services (Git hosting, media server, web applications)
- **Reverse Proxy** - External gateway with SSL termination and service routing
### The Stack
- **OS**: NixOS 25.05 (Warbler) - because reproducible builds are beautiful
- **Configuration**: Nix Flakes with modular approach - keeping things organized
- **Virtualization**: Incus, Libvirt/QEMU, Podman - gotta test stuff somewhere
- **Desktop**: GNOME, Cosmic, Sway - variety is the spice of life
- **Storage**: ZFS with snapshots and NFS - never lose data again
- **Network**: Tailscale mesh - because VPNs should just work
### Technology Stack
- **Base OS**: NixOS 25.05 with Nix Flakes
- **Configuration**: Modular, declarative system configuration
- **Virtualization**: Incus containers, Libvirt/QEMU VMs, Podman containers
- **Desktop**: GNOME, Cosmic, Sway window managers
- **Storage**: ZFS with snapshots, automated mounting, NFS network storage
- **Network**: Tailscale mesh VPN with centralized hostname resolution
## 📁 How It's Organized
## Project Structure
Everything's broken down into logical chunks to keep things manageable:
Modular configuration organized for scalability and maintainability:
```
Home-lab/
├── flake.nix # Main flake configuration
├── flake.lock # Locked dependency versions
├── flake.lock # Dependency lock file
├── machines/ # Machine-specific configurations
│ ├── congenital-optimist/ # AMD workstation
│ └── sleeper-service/ # Intel file server
│ ├── workstation/ # Development machine config
│ ├── file-server/ # NFS storage server
│ ├── app-server/ # Containerized services
│ └── reverse-proxy/ # External gateway
├── modules/ # Reusable NixOS modules
│ ├── common/ # Shared system configuration
│ ├── common/ # Base system configuration
│ ├── desktop/ # Desktop environment modules
│ ├── development/ # Development tools and editors
│ ├── hardware/ # Hardware-specific configurations
│ ├── development/ # Development tools
│ ├── services/ # Service configurations
│ ├── system/ # Core system modules
│ ├── users/ # User configurations
│ ├── users/ # User management
│ └── virtualization/ # Container and VM setup
├── users/ # User-specific configurations
│ └── geir/ # Primary user configuration
│ ├── dotfiles/ # Literate configuration with org-mode
│ └── user.nix # System-level user config
├── overlays/ # Nix package overlays
├── packages/ # Custom package definitions
└── secrets/ # Encrypted secrets (future)
├── packages/ # Custom packages and tools
└── research/ # Documentation and analysis
```
## 🔧 How I Manage This Chaos
## Configuration Philosophy
### Keeping Things Modular
I've split everything into focused modules so I don't go insane:
### Modular Design
- **Single Responsibility**: Each module handles one aspect of system configuration
- **Composable**: Modules can be mixed and matched per machine requirements
- **Testable**: Individual modules can be validated independently
- **Documented**: Clear documentation for module purpose and configuration
- **Desktop Environments**: Each DE gets its own module - no more giant config files
- **Virtualization**: Separate configs for Incus, Libvirt, and Podman - mix and match as needed
- **Development**: Modular tool setups for different workflows - because context switching is real
- **Hardware**: Hardware-specific tweaks and drivers - make the silicon sing
### User Management Strategy
- **Role-based Users**: Separate users for desktop vs server administration
- **Centralized Configuration**: Consistent user setup across all machines
- **Security Focus**: SSH key management and privilege separation
- **Literate Dotfiles**: Org-mode documentation for complex configurations
### Literate Programming (Because Documentation Matters)
My user configs live in Emacs org-mode files - it's like having your documentation and code hold hands:
- Configuration files that explain themselves
- Automatic tangling from `.org` files to actual configs
- Git tracks both the code and the reasoning behind it
### Network Architecture
- **Mesh VPN**: Tailscale for secure inter-machine communication
- **Service Discovery**: Centralized hostname resolution
- **Firewall Management**: Service-specific port configuration
- **External Access**: Reverse proxy with SSL termination
## 🚀 My Workflow
## Development Workflow
### Tinkering Locally
### Local Testing
```bash
# Check if I broke anything
# Validate configuration syntax
nix flake check
# Test changes without committing to them
sudo nixos-rebuild test --flake .#<machine-name>
# Build without applying changes
nix build .#nixosConfigurations.<machine>.config.system.build.toplevel
# Build and see what happens
sudo nixos-rebuild build --flake .#<machine-name>
# Test configuration (temporary)
sudo nixos-rebuild test --flake .#<machine>
# Ship it!
sudo nixos-rebuild switch --flake .#<machine-name>
# Apply configuration permanently
sudo nixos-rebuild switch --flake .#<machine>
```
### Git-Driven Chaos (In a Good Way)
1. **Feature Branch**: New idea? New branch.
2. **Local Testing**: Break things safely with `nixos-rebuild test`
3. **Pull Request**: Show off the changes
4. **Review**: Someone sanity-checks my work
5. **Deploy**: Either automated or "click the button"
### Git Workflow
1. **Feature Branch**: Create branch for configuration changes
2. **Local Testing**: Validate changes with `nixos-rebuild test`
3. **Pull Request**: Submit changes for review
4. **Deploy**: Apply configuration to target machines
## 🔐 Secrets and Security
### Remote Deployment
- **SSH-based**: Remote deployment via secure shell
- **Atomic Updates**: Complete success or automatic rollback
- **Health Checks**: Service validation after deployment
- **Centralized Management**: Single repository for all infrastructure
### Current Reality
- No secrets in git (obviously)
- Manual secret juggling during setup (it's fine, really)
- ZFS encryption for the important stuff
## Service Architecture
### Future Dreams
- **agenix** or **sops-nix** for proper secret management
- **age** keys for encryption magic
- **CI/CD** that doesn't leak passwords everywhere
### Core Services
- **Git Hosting**: Self-hosted Git with CI/CD capabilities
- **Media Server**: Streaming with transcoding support
- **File Storage**: NFS network storage with ZFS snapshots
- **Web Gateway**: Reverse proxy with SSL and external access
- **Container Platform**: Podman for containerized applications
## 🎯 The Hardware
### Service Discovery
- **Internal DNS**: Tailscale for mesh network resolution
- **External DNS**: Public domain with SSL certificates
- **Service Mesh**: Inter-service communication via secure network
- **Load Balancing**: Traffic distribution and failover
### CongenitalOptimist (The Workstation)
- **CPU**: AMD Threadripper (check hardware-configuration.nix for the gory details)
- **GPU**: AMD (with proper drivers and GPU passthrough for VMs)
- **Storage**: ZFS pools (zpool for system, stuffpool for data hoarding)
- **Role**: Main development machine, VM playground, desktop environment testing ground
- **Services**: Whatever I'm experimenting with this week
### Data Management
- **ZFS Storage**: Copy-on-write filesystem with snapshots
- **Network Shares**: NFS for cross-machine file access
- **Backup Strategy**: Automated snapshots and external backup
- **Data Integrity**: Checksums and redundancy
### SleeperService (The Quiet One)
- **CPU**: Intel Xeon E3-1230 V2 @ 3.70GHz (4 cores, 8 threads - still plenty peppy)
- **Memory**: 16GB RAM (enough for file serving duties)
- **Storage**: ZFS with redundancy (because data loss is sadness)
- **Role**: Network storage, file sharing, backup duties, monitoring the other machines
- **Services**: NFS, Samba, automated backups, keeping an eye on things
## Security Model
## 🧪 Testing (The "Does It Work?" Phase)
### Network Security
- **VPN Mesh**: All inter-machine traffic via Tailscale
- **Firewall Rules**: Service-specific port restrictions
- **SSH Hardening**: Key-based authentication only
- **Fail2ban**: Automated intrusion prevention
### Automated Testing (Someday Soon)
- **Configuration Validation**: `nix flake check` in CI - catch dumb mistakes early
- **Build Testing**: Test builds for all machines - make sure nothing's broken
- **Module Testing**: Individual module validation - each piece should work alone
- **Integration Testing**: Full system builds - the moment of truth
### User Security
- **Role Separation**: Administrative vs daily-use accounts
- **Key Management**: Centralized SSH key distribution
- **Privilege Escalation**: Sudo access only where needed
- **Service Accounts**: Dedicated accounts for automated services
### My Manual Testing Ritual
- [ ] System actually boots (surprisingly important)
- [ ] Desktop environments don't crash immediately
- [ ] VMs and containers start up
- [ ] Network services respond
- [ ] Development environment loads
- [ ] Can actually get work done
### Infrastructure Security
- **Configuration as Code**: All changes tracked in version control
- **Atomic Deployments**: Rollback capability for failed changes
- **Secret Management**: Encrypted secrets with controlled access
- **Security Updates**: Regular dependency updates
## 📈 Keeping Things Running
## Testing Strategy
### Health Checks (The Boring But Important Stuff)
- Generation switching (did the update work?)
- Service status monitoring (what's broken now?)
- ZFS pool health (happy disks = happy life)
- Network connectivity (can I reach the internet?)
- Resource usage (is something eating all my RAM?)
### Automated Testing
- **Syntax Validation**: Nix flake syntax checking
- **Build Testing**: Configuration build verification
- **Module Testing**: Individual component validation
- **Integration Testing**: Full system deployment tests
### Backup Strategy (Paranoia Pays Off)
- **ZFS Snapshots**: Automatic filesystem snapshots - time travel for your data
- **Configuration Backups**: Git repo with full history - every mistake preserved for posterity
- **Data Backups**: Automated services on SleeperService - redundancy is key
- **Recovery Procedures**: Documented rollback processes - for when everything goes sideways
### Manual Testing
- **Boot Validation**: System startup verification
- **Service Health**: Application functionality checks
- **Network Connectivity**: Inter-service communication tests
- **User Environment**: Desktop and development tool validation
## 🔄 CI/CD Dreams (Work in Progress)
## Deployment Status
### Validation Pipeline (The Plan)
```yaml
# What I want GitHub Actions to do
- Syntax Check: nix flake check # Catch the obvious stuff
- Build Test: nix build .#nixosConfigurations.<machine> # Does it actually build?
- Security Scan: Nix security auditing # Keep the bad guys out
- Documentation: Update system docs # Because future me will forget
```
### Infrastructure Maturity
- ✅ **Multi-machine Configuration**: 4 machines deployed
- ✅ **Service Integration**: Git hosting, media server, file storage
- ✅ **Network Mesh**: Secure VPN with service discovery
- ✅ **External Access**: Public services with SSL termination
- ✅ **Centralized Management**: Single repository for all infrastructure
### Deployment Pipeline (The Dream)
```yaml
# Automated deployment magic
- Staging: Deploy to test environment # Break things safely
- Integration Tests: Automated system testing # Does everything still work?
- Production: Deploy to production machines # The moment of truth
- Rollback: Automatic rollback on failure # When things go wrong (they will)
```
### Current Capabilities
- **Development Environment**: Full IDE setup with multiple desktop options
- **File Services**: Network storage with 900GB+ media library
- **Git Hosting**: Self-hosted with external access
- **Media Streaming**: Movie and TV series streaming with transcoding
- **Container Platform**: Podman-based containerized services
## 🤝 Want to Contribute?
## Documentation
### How to Jump In
1. Fork or clone the repo
2. Create a feature branch for your idea
3. Make your changes
4. Test locally with `nixos-rebuild test` (don't break my machine)
5. Submit a pull request
6. Chat about it in the review
7. Merge when we're both happy
- **[Migration Plan](plan.md)**: Detailed implementation roadmap
- **[Development Workflow](DEVELOPMENT_WORKFLOW.md)**: Contribution guidelines
- **[Branching Strategy](BRANCHING_STRATEGY.md)**: Git workflow and conventions
- **[AI Instructions](instruction.md)**: Agent guidance for system management
### Module Development Tips
- Keep modules focused - one job, do it well
- Document what your module does and how to use it
- Test modules independently when you can
- Use consistent naming (future you will thank you)
- Include example configurations for others
## Contributing
## 📖 Documentation
### Getting Started
1. Fork the repository
2. Create feature branch
3. Test changes locally with `nixos-rebuild test`
4. Submit pull request with detailed description
5. Respond to review feedback
6. Deploy after approval
- **[Plan](plan.md)**: The grand vision and migration roadmap
- **[Instructions](instruction.md)**: Step-by-step setup and AI agent guidance
- **[Machine Documentation](machines/)**: Individual machine configs and notes
- **[Module Documentation](modules/)**: How each module works
- **[User Documentation](users/)**: User-specific configuration details
### Module Development
- **Focused Scope**: One responsibility per module
- **Configuration Options**: Parameterize for flexibility
- **Documentation**: Explain purpose and usage
- **Examples**: Provide usage examples
## 🎯 The Journey So Far
## License
### Phase 1: Flakes Migration ✅
- [x] Converted to flake-based configuration (no more channels!)
- [x] Modularized desktop environments (sanity preserved)
- [x] Added comprehensive virtualization (all the containers)
- [x] Set up GitOps foundation (git-driven everything)
### Phase 2: Configuration Cleanup (In Progress)
- [ ] Optimize modular structure (make it even better)
- [ ] Enhance documentation (explain the magic)
- [ ] Standardize module interfaces (consistency is king)
### Phase 3: Multi-Machine Expansion (Coming Soon)
- [ ] Add SleeperService configuration (wake up the sleeper)
- [ ] Implement service modules (automate all the things)
- [ ] Set up network storage (centralized data paradise)
### Phase 4: Automation & CI/CD (The Dream)
- [ ] Implement automated testing (catch problems early)
- [ ] Set up deployment pipelines (one-click deploys)
- [ ] Add monitoring and alerting (know when things break)
### Phase 5: Advanced Features (Future Fun)
- [ ] Secrets management (proper secret handling)
- [ ] Advanced monitoring (graphs and dashboards)
- [ ] Backup automation (paranoia made easy)
## 📄 License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. Feel free to steal ideas, improve things, or just poke around.
## 🙏 Thanks
- **NixOS Community** for excellent docs and endless patience with newbie questions
- **Culture Ship Names** for inspiring machine nomenclature (because why not?)
- **Emacs Community** for literate programming inspiration and org-mode magic
- **Home Lab Community** for sharing knowledge, war stories, and "it works on my machine" solutions
MIT License - see [LICENSE](LICENSE) for details.
---
*"The ship had decided to call itself the Arbitrary, presumably for much the same reason."*
*Infrastructure designed for reliability, security, and maintainability.*

257
README_old.md Normal file
View file

@ -0,0 +1,257 @@
# NixOS Home Lab Infrastructure
[![NixOS](https://img.shields.io/badge/NixOS-25.05-blue.svg)](https://nixos.org/)
[![Flakes](https://img.shields.io/badge/Nix-Flakes-green.svg)](https://nixos.wiki/wiki/Flakes)
[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
Modular NixOS flake configuration for multi-machine home lab infrastructure. Features declarative system configuration, centralized user management, and scalable service deployment across development workstations and server infrastructure.
## Quick Start
```bash
# Clone repository
git clone <repository-url> Home-lab
cd Home-lab
# Validate configuration
nix flake check
# Test configuration (temporary, reverts on reboot)
sudo nixos-rebuild test --flake .#<machine-name>
# Apply configuration permanently
sudo nixos-rebuild switch --flake .#<machine-name>
```
## Architecture Overview
### Machine Types
- **Development Workstation** - High-performance AMD Threadripper with desktop environments
- **File Server** - Intel Xeon with ZFS storage and NFS services
- **Application Server** - Containerized services (Git hosting, media server)
- **Reverse Proxy** - External gateway with SSL termination and service routing
### Technology Stack
- **Base OS**: NixOS 25.05 with Nix Flakes
- **Configuration**: Modular, declarative system configuration
- **Virtualization**: Incus containers, Libvirt/QEMU VMs, Podman containers
- **Desktop**: GNOME, Cosmic, Sway window managers
- **Storage**: ZFS with snapshots, automated mounting, NFS network storage
- **Network**: Tailscale mesh VPN with centralized hostname resolution
## 📁 How It's Organized
Everything's broken down into logical chunks to keep things manageable:
```
Home-lab/
├── flake.nix # Main flake configuration
├── flake.lock # Locked dependency versions
├── machines/ # Machine-specific configurations
│ ├── congenital-optimist/ # AMD workstation
│ └── sleeper-service/ # Intel file server
├── modules/ # Reusable NixOS modules
│ ├── common/ # Shared system configuration
│ ├── desktop/ # Desktop environment modules
│ ├── development/ # Development tools and editors
│ ├── hardware/ # Hardware-specific configurations
│ ├── services/ # Service configurations
│ ├── system/ # Core system modules
│ ├── users/ # User configurations
│ └── virtualization/ # Container and VM setup
├── users/ # User-specific configurations
│ └── geir/ # Primary user configuration
│ ├── dotfiles/ # Literate configuration with org-mode
│ └── user.nix # System-level user config
├── overlays/ # Nix package overlays
├── packages/ # Custom package definitions
└── secrets/ # Encrypted secrets (future)
```
## 🔧 How I Manage This Chaos
### Keeping Things Modular
I've split everything into focused modules so I don't go insane:
- **Desktop Environments**: Each DE gets its own module - no more giant config files
- **Virtualization**: Separate configs for Incus, Libvirt, and Podman - mix and match as needed
- **Development**: Modular tool setups for different workflows - because context switching is real
- **Hardware**: Hardware-specific tweaks and drivers - make the silicon sing
### Literate Programming (Because Documentation Matters)
My user configs live in Emacs org-mode files - it's like having your documentation and code hold hands:
- Configuration files that explain themselves
- Automatic tangling from `.org` files to actual configs
- Git tracks both the code and the reasoning behind it
## 🚀 My Workflow
### Tinkering Locally
```bash
# Check if I broke anything
nix flake check
# Test changes without committing to them
sudo nixos-rebuild test --flake .#<machine-name>
# Build and see what happens
sudo nixos-rebuild build --flake .#<machine-name>
# Ship it!
sudo nixos-rebuild switch --flake .#<machine-name>
```
### Git-Driven Chaos (In a Good Way)
1. **Feature Branch**: New idea? New branch.
2. **Local Testing**: Break things safely with `nixos-rebuild test`
3. **Pull Request**: Show off the changes
4. **Review**: Someone sanity-checks my work
5. **Deploy**: Either automated or "click the button"
## 🔐 Secrets and Security
### Current Reality
- No secrets in git (obviously)
- Manual secret juggling during setup (it's fine, really)
- ZFS encryption for the important stuff
### Future Dreams
- **agenix** or **sops-nix** for proper secret management
- **age** keys for encryption magic
- **CI/CD** that doesn't leak passwords everywhere
## 🎯 The Hardware
### CongenitalOptimist (The Workstation)
- **CPU**: AMD Threadripper (check hardware-configuration.nix for the gory details)
- **GPU**: AMD (with proper drivers and GPU passthrough for VMs)
- **Storage**: ZFS pools (zpool for system, stuffpool for data hoarding)
- **Role**: Main development machine, VM playground, desktop environment testing ground
- **Services**: Whatever I'm experimenting with this week
### SleeperService (The Quiet One)
- **CPU**: Intel Xeon E3-1230 V2 @ 3.70GHz (4 cores, 8 threads - still plenty peppy)
- **Memory**: 16GB RAM (enough for file serving duties)
- **Storage**: ZFS with redundancy (because data loss is sadness)
- **Role**: Network storage, file sharing, backup duties, monitoring the other machines
- **Services**: NFS, Samba, automated backups, keeping an eye on things
## 🧪 Testing (The "Does It Work?" Phase)
### Automated Testing (Someday Soon)
- **Configuration Validation**: `nix flake check` in CI - catch dumb mistakes early
- **Build Testing**: Test builds for all machines - make sure nothing's broken
- **Module Testing**: Individual module validation - each piece should work alone
- **Integration Testing**: Full system builds - the moment of truth
### My Manual Testing Ritual
- [ ] System actually boots (surprisingly important)
- [ ] Desktop environments don't crash immediately
- [ ] VMs and containers start up
- [ ] Network services respond
- [ ] Development environment loads
- [ ] Can actually get work done
## 📈 Keeping Things Running
### Health Checks (The Boring But Important Stuff)
- Generation switching (did the update work?)
- Service status monitoring (what's broken now?)
- ZFS pool health (happy disks = happy life)
- Network connectivity (can I reach the internet?)
- Resource usage (is something eating all my RAM?)
### Backup Strategy (Paranoia Pays Off)
- **ZFS Snapshots**: Automatic filesystem snapshots - time travel for your data
- **Configuration Backups**: Git repo with full history - every mistake preserved for posterity
- **Data Backups**: Automated services on SleeperService - redundancy is key
- **Recovery Procedures**: Documented rollback processes - for when everything goes sideways
## 🔄 CI/CD Dreams (Work in Progress)
### Validation Pipeline (The Plan)
```yaml
# What I want GitHub Actions to do
- Syntax Check: nix flake check # Catch the obvious stuff
- Build Test: nix build .#nixosConfigurations.<machine> # Does it actually build?
- Security Scan: Nix security auditing # Keep the bad guys out
- Documentation: Update system docs # Because future me will forget
```
### Deployment Pipeline (The Dream)
```yaml
# Automated deployment magic
- Staging: Deploy to test environment # Break things safely
- Integration Tests: Automated system testing # Does everything still work?
- Production: Deploy to production machines # The moment of truth
- Rollback: Automatic rollback on failure # When things go wrong (they will)
```
## 🤝 Want to Contribute?
### How to Jump In
1. Fork or clone the repo
2. Create a feature branch for your idea
3. Make your changes
4. Test locally with `nixos-rebuild test` (don't break my machine)
5. Submit a pull request
6. Chat about it in the review
7. Merge when we're both happy
### Module Development Tips
- Keep modules focused - one job, do it well
- Document what your module does and how to use it
- Test modules independently when you can
- Use consistent naming (future you will thank you)
- Include example configurations for others
## 📖 Documentation
- **[Plan](plan.md)**: The grand vision and migration roadmap
- **[Instructions](instruction.md)**: Step-by-step setup and AI agent guidance
- **[Machine Documentation](machines/)**: Individual machine configs and notes
- **[Module Documentation](modules/)**: How each module works
- **[User Documentation](users/)**: User-specific configuration details
## 🎯 The Journey So Far
### Phase 1: Flakes Migration ✅
- [x] Converted to flake-based configuration (no more channels!)
- [x] Modularized desktop environments (sanity preserved)
- [x] Added comprehensive virtualization (all the containers)
- [x] Set up GitOps foundation (git-driven everything)
### Phase 2: Configuration Cleanup (In Progress)
- [ ] Optimize modular structure (make it even better)
- [ ] Enhance documentation (explain the magic)
- [ ] Standardize module interfaces (consistency is king)
### Phase 3: Multi-Machine Expansion (Coming Soon)
- [ ] Add SleeperService configuration (wake up the sleeper)
- [ ] Implement service modules (automate all the things)
- [ ] Set up network storage (centralized data paradise)
### Phase 4: Automation & CI/CD (The Dream)
- [ ] Implement automated testing (catch problems early)
- [ ] Set up deployment pipelines (one-click deploys)
- [ ] Add monitoring and alerting (know when things break)
### Phase 5: Advanced Features (Future Fun)
- [ ] Secrets management (proper secret handling)
- [ ] Advanced monitoring (graphs and dashboards)
- [ ] Backup automation (paranoia made easy)
## 📄 License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. Feel free to steal ideas, improve things, or just poke around.
## 🙏 Thanks
- **NixOS Community** for excellent docs and endless patience with newbie questions
- **Culture Ship Names** for inspiring machine nomenclature (because why not?)
- **Emacs Community** for literate programming inspiration and org-mode magic
- **Home Lab Community** for sharing knowledge, war stories, and "it works on my machine" solutions
---
*"The ship had decided to call itself the Arbitrary, presumably for much the same reason."*

View file

@ -60,6 +60,18 @@
./modules/common/tty.nix
];
};
# grey-area - Services host (Forgejo, Jellyfin, etc.)
grey-area = nixpkgs.lib.nixosSystem {
inherit system specialArgs;
modules = [
./machines/grey-area/configuration.nix
./machines/grey-area/hardware-configuration.nix
./modules/common/nix.nix
./modules/common/base.nix
./modules/common/tty.nix
];
};
};
# Custom packages for the home lab
@ -83,6 +95,7 @@
echo " - congenital-optimist (Threadripper workstation)"
echo " - sleeper-service (Xeon file server)"
echo " - reverse-proxy (VPS edge server)"
echo " - grey-area (Services host: Forgejo, Jellyfin, etc.)"
echo ""
echo "Build with: nixos-rebuild build --flake .#<config>"
echo "Switch with: nixos-rebuild switch --flake .#<config>"

View file

@ -89,6 +89,19 @@ Home-lab/
- **Status monitoring**: Use `lab status` to get overview of all lab machines
- **Document Context7 findings** in commit messages
### Git/Forgejo Configuration
- **Primary repository**: Hosted on self-hosted Forgejo instance
- **Forgejo URL**: `ssh://forgejo@git.geokkjer.eu:1337/geir/Home-lab.git`
- **SSH port**: 1337 (proxied through reverse-proxy to grey-area:22)
- **User**: Must use `forgejo` user, not `git` user
- **GitHub mirror**: `git@github.com:geokkjer/Home-lab.git` (secondary/backup)
- **Remote configuration**:
```bash
git remote add origin ssh://forgejo@git.geokkjer.eu:1337/geir/Home-lab.git
git remote add github git@github.com:geokkjer/Home-lab.git
```
- **Pushing**: Primary pushes to Forgejo origin, manual sync to GitHub as needed
## Key Constraints
- **No Home Manager**: Use org-mode literate dotfiles instead
- **ZFS preservation**: Never change hostId or break ZFS mounts

View file

@ -12,6 +12,9 @@
# Security modules
../../modules/security/ssh-keys.nix
# Network modules
../../modules/network/extraHosts.nix
# Hardware modules
../../modules/hardware/amd-workstation.nix
@ -45,8 +48,7 @@
path = "/boot";
}
];
};
# ZFS services for this machine
}; # ZFS services for this machine
services.zfs = {
autoScrub.enable = true;
trim.enable = true;

View file

@ -1,21 +1,26 @@
{ config, pkgs, ... }:
{
imports =
[ # Include the results of the hardware scan.
./hardware-configuration.nix
./starship.nix
./aliases.nix
./podman.nix
./libvirt.nix
./incus.nix
./jellyfin.nix
./tailscale.nix
./calibre-web.nix
./audiobook.nix
#./ollama.nix
./forgejo.nix
];
imports = [
# Hardware configuration
./hardware-configuration.nix
# Shared modules
../../modules/common/base.nix
../../modules/network/common.nix
../../modules/network/extraHosts.nix
../../modules/virtualization/podman.nix
../../modules/virtualization/libvirt.nix
../../modules/virtualization/incus.nix
../../modules/users/sma.nix
# Services
./services/jellyfin.nix
./services/calibre-web.nix
./services/audiobook.nix
./services/forgejo.nix
#./services/ollama.nix
];
# Swap zram
zramSwap = {
@ -34,9 +39,20 @@
# Mount remote filesystem
fileSystems."/mnt/remote/media" = {
device = "sleeper-service:/mnt/storage";
device = "sleeper-service:/mnt/storage/media";
fsType = "nfs";
options = [ "x-systemd.automount" ];
options = [
"x-systemd.automount"
"x-systemd.idle-timeout=60"
"x-systemd.device-timeout=10"
"x-systemd.mount-timeout=10"
"noauto"
"soft"
"intr"
"timeo=10"
"retrans=3"
"_netdev"
];
};
# Enable all unfree hardware support.
@ -48,11 +64,16 @@
# Networking
networking.hostName = "grey-area";
networking.networkmanager.enable = true;
networking.networkmanager.enable = true;
# Set your time zone.
time.timeZone = "Europe/Oslo";
# Text mode configuration (headless server)
services.xserver.enable = false;
services.displayManager.defaultSession = "none";
boot.kernelParams = [ "systemd.unit=multi-user.target" ];
systemd.targets.graphical.enable = false;
i18n.defaultLocale = "en_US.UTF-8";
console = {
@ -60,19 +81,6 @@
keyMap = "no";
};
users.users.geir = {
isNormalUser = true;
extraGroups = [ "wheel"
"networkmanager"
"libvirt"
"podman"
"incus-admin"
];
packages = with pkgs; [
bottom fastfetch nerdfetch
];
};
environment.systemPackages = with pkgs; [
neovim emacs nano curl htop glances kitty
wget git inxi nethogs fastfetch
@ -84,12 +92,9 @@
services.openssh.settings.PasswordAuthentication = true;
# Enable Netdata
services.netdata.enable = true;
# Firewall
networking.firewall.enable = true;
networking.firewall.allowedTCPPorts = [ 22 19999 23231];
networking.firewall.allowedTCPPorts = [ 22 3000 23231];
networking.firewall.allowedUDPPorts = [ 22 23231 ];
networking.nftables.enable = true;
system.stateVersion = "23.05"; # Do not change this, it maintains data compatibility.

View file

@ -22,10 +22,7 @@
{ device = "/dev/disk/by-uuid/E251-F60A";
fsType = "vfat";
};
fileSystems."/mnt/remote/media" =
{ device = "sleeper-service:/mnt/storage";
fsType = "nfs";
};
swapDevices = [ ];
# Enables DHCP on each ethernet and wireless interface. In case of scripted networking
# (the default) this is the recommended approach. When using systemd-networkd it's

View file

@ -1,20 +0,0 @@
{ config, pkgs, ... }:
{
environment.systemPackages = with pkgs; [
tldr
eza
bat
ripgrep
];
environment.shellAliases = {
vi = "nvim";
vim = "nvim";
h = "tldr";
# oxidized
ls = "eza -l";
cat = "bat";
grep = "rg";
top = "btm --color gruvbox";
# some tools
};
}

View file

@ -1,14 +0,0 @@
{ config, pkgs, ... }:
{
virtualisation.incus = {
enable = true;
ui.enable = true;
package = pkgs.incus;
};
environment.systemPackages = with pkgs; [
incus
lxc
];
networking.firewall.allowedTCPPorts = [ 8443 ];
}

View file

@ -1,7 +0,0 @@
{ config, pkgs, lib, ... }:
{
environment.systemPackages = with pkgs;
[
kitty kitty-themes termpdfpy
];
}

View file

@ -1,9 +0,0 @@
{ config, pkgs, ... }:
{
virtualisation.libvirtd.enable = true;
environment.systemPackages = with pkgs; [
qemu_kvm
libvirt
polkit
];
}

View file

@ -1,30 +0,0 @@
{ pkgs, ... }:
{
# Nextcloud Config
environment.etc."nextcloud-admin-pass".text = "siKKerhet666";
services.nextcloud = {
enable = true;
hostName = "server1.tail807ea.ts.net";
# Ssl Let'encrypt
#hostName = "cloud.geokkjer.eu";
#https = true;
# Auto-update Nextcloud Apps
autoUpdateApps.enable = true;
# Set what time makes sense for you
autoUpdateApps.startAt = "05:00:00";
# enable redis cache
configureRedis = true;
# Create db locally , maybe not needed with sqlite
database.createLocally = true;
# Config options
config = {
dbtype = "sqlite";
adminpassFile = "/etc/nextcloud-admin-pass";
trustedProxies = [ "46.226.104.98" "100.75.29.52" ];
extraTrustedDomains = [ "localhost" "*.cloudflare.net" "*.tail807ea.ts.net" "46.226.104.98" "*.geokkjer.eu" ];
};
};
}

View file

@ -1,9 +0,0 @@
{ pkgs, configs, ... }:
{
services.openvscode-server = {
enable = true;
telemetryLevel = "off";
port = 3003;
host = "0.0.0.0";
};
}

View file

@ -1,13 +0,0 @@
{ config, pkgs, ... }:
{
virtualisation.podman.enable = true;
virtualisation.podman.dockerCompat = true;
virtualisation.podman.dockerSocket.enable = true;
#virtualisation.defaultNetwork.settings.dns_enabled = true;
environment.systemPackages = with pkgs; [
podman-tui
podman-compose
];
}

View file

@ -1,6 +0,0 @@
{ pkgs, ... }:
{
environment.systemPackages = with pkgs; [
starship
];
}

View file

@ -1,18 +0,0 @@
{config, pkgs, ... }:
{
environment.systemPackages = with pkgs; [
tailscale
];
services.tailscale.enable = true;
networking.firewall = {
# trace: warning: Strict reverse path filtering breaks Tailscale exit node
# use and some subnet routing setups. Consider setting
# `networking.firewall.checkReversePath` = 'loose'
checkReversePath = "loose";
trustedInterfaces = [ "tailscale0" ];
};
}
#+end80808080 * nginx
#+begin_src nix

View file

@ -1,29 +0,0 @@
{ pkgs, ... }:
{
services.getty.greetingLine = ''\l'';
console = {
earlySetup = true;
# Joker palette
colors = [
"1b161f"
"ff5555"
"54c6b5"
"d5aa2a"
"bd93f9"
"ff79c6"
"8be9fd"
"bfbfbf"
"1b161f"
"ff6e67"
"5af78e"
"ffce50"
"caa9fa"
"ff92d0"
"9aedfe"
"e6e6e6"
];
};
}

View file

@ -1,26 +0,0 @@
{ pkgs, configs, ... }:
{
services.writefreely = {
enable = true;
admin.name = "geir@geokkjer.eu";
host = "blog.geokkjer.eu";
database = {
type = "sqlite3";
#filename = "writefreely.db";
#database = "writefreely";
};
nginx = {
# Enable Nginx and configure it to serve WriteFreely.
enable = true;
};
settings = {
server = {
port = 8088;
bind = "0.0.0.0";
};
};
};
networking.firewall.allowedTCPPorts = [ 8088 ];
networking.firewall.allowedUDPPorts = [ 8088 ];
}

View file

@ -13,9 +13,9 @@
DISABLE_REGISTRATION = true;
};
server = {
ROOT_URL = "http://apps:3000";
ROOT_URL = "https://git.geokkjer.eu";
SSH_DOMAIN = "git.geokkjer.eu";
SSH_PORT = 2222;
SSH_PORT = 1337;
};
repository = {
ENABLE_PUSH_CREATE_USER = true;

View file

@ -4,6 +4,7 @@
imports = [
./gandicloud.nix
../../modules/common/base.nix
../../modules/network/extraHosts.nix
../../modules/users/sma.nix
../../modules/security/ssh-keys.nix
];
@ -19,8 +20,8 @@
# DMZ-specific firewall configuration - very restrictive
networking.firewall = {
enable = true;
# Allow HTTP/HTTPS from external network and Git SSH on port 2222
allowedTCPPorts = [ 80 443 2222 ];
# Allow HTTP/HTTPS from external network and Git SSH on port 1337
allowedTCPPorts = [ 80 443 1337 ];
allowedUDPPorts = [ ];
# SSH only allowed from Tailscale network (100.64.0.0/10)
extraCommands = ''
@ -71,7 +72,7 @@
"git.geokkjer.eu" = {
addSSL = true;
enableACME = true;
locations."/".proxyPass = "http://apps:3000";
locations."/".proxyPass = "http://grey-area:3000";
};
#"geokkjer.eu" = {
# default = true;
@ -84,11 +85,11 @@
# Stream configuration for SSH forwarding to Git server
streamConfig = ''
upstream git_ssh_backend {
server apps:22;
server grey-area:22;
}
server {
listen 2222;
listen 1337;
proxy_pass git_ssh_backend;
proxy_timeout 300s;
proxy_connect_timeout 10s;

View file

@ -5,6 +5,7 @@
../../modules/security/ssh-keys.nix
# Network configuration
./network-sleeper-service.nix
../../modules/network/extraHosts.nix
# Services
./nfs.nix
./services/transmission.nix
@ -57,10 +58,10 @@
# ];
# Create mount directories early in boot process
systemd.tmpfiles.rules = [
"d /mnt/storage 0755 root root -"
"d /mnt/storage/media 0755 root root -"
];
# systemd.tmpfiles.rules = [
# "d /mnt/storage 0755 root root -"
# "d /mnt/storage/media 0755 root root -"
# ];
# Network configuration - using working setup from old config
# networking.hostName = "sleeper-service";

View file

@ -41,21 +41,26 @@
# nameservers = [ "10.0.0.14" "10.0.0.138" "8.8.8.8" ]; # Pi-hole, router, Google DNS fallback
# Additional firewall ports for file server services
firewall.allowedTCPPorts = [
22 # SSH
111 # NFS portmapper
2049 # NFS
445 # SMB/CIFS
139 # NetBIOS Session Service
# Add additional ports here as needed
];
firewall = {
# Trust the Tailscale interface for mesh network access
trustedInterfaces = [ "tailscale0" ];
allowedTCPPorts = [
22 # SSH
111 # NFS portmapper
2049 # NFS
445 # SMB/CIFS
139 # NetBIOS Session Service
# Add additional ports here as needed
];
firewall.allowedUDPPorts = [
22 # SSH
111 # NFS portmapper
2049 # NFS
137 # NetBIOS Name Service
138 # NetBIOS Datagram Service
];
allowedUDPPorts = [
22 # SSH
111 # NFS portmapper
2049 # NFS
137 # NetBIOS Name Service
138 # NetBIOS Datagram Service
];
};
};
}

View file

@ -7,20 +7,22 @@
services.nfs.server = {
enable = true;
# Export the storage directory (ZFS dataset)
# Allow access from both local network and Tailscale network
exports = ''
/mnt/storage 10.0.0.0/24(rw,sync,no_subtree_check,no_root_squash)
/mnt/storage 10.0.0.0/24(rw,sync,no_subtree_check,no_root_squash) 100.64.0.0/10(rw,sync,no_subtree_check,no_root_squash)
/mnt/storage/media 10.0.0.0/24(rw,sync,no_subtree_check,no_root_squash) 100.64.0.0/10(rw,sync,no_subtree_check,no_root_squash)
'';
# Create exports on startup
createMountPoints = true;
};
# Ensure the storage subdirectories exist (ZFS dataset is mounted at /mnt/storage)
systemd.tmpfiles.rules = [
"d /mnt/storage/media 0755 sma users -"
"d /mnt/storage/downloads 0755 sma users -"
"d /mnt/storage/backups 0755 sma users -"
"d /mnt/storage/shares 0755 sma users -"
];
# systemd.tmpfiles.rules = [
# "d /mnt/storage/media 0755 sma users -"
# "d /mnt/storage/downloads 0755 sma users -"
# "d /mnt/storage/backups 0755 sma users -"
# "d /mnt/storage/shares 0755 sma users -"
# ];
# Required packages for NFS
environment.systemPackages = with pkgs; [

View file

@ -0,0 +1,26 @@
# Network hostname resolution module
# Provides consistent hostname-to-IP mapping across all home lab machines
# Uses Tailscale IPs for reliable connectivity across the mesh network
{ config, lib, ... }:
{
# Add hostname entries for all home lab machines using Tailscale IPs
networking.extraHosts = ''
# Home Lab Infrastructure (Tailscale mesh network)
100.109.28.53 congenital-optimist
100.81.15.84 sleeper-service
100.119.86.92 grey-area
100.96.189.104 reverse-proxy vps1
# Additional network devices
100.103.143.108 pihole
100.126.202.40 wordpresserver
'';
# Enable Tailscale by default for all machines using this module
services.tailscale = {
enable = true;
useRoutingFeatures = "client";
};
}

View file

@ -51,11 +51,11 @@
User geir
IdentityFile ~/.ssh/id_ed25519_dev
Host grey-area grey-area.home 10.0.0.11
Host grey-area grey-area.home 10.0.0.12
User geir
IdentityFile ~/.ssh/id_ed25519_dev
Host reverse-proxy reverse-proxy.home 10.0.0.12
Host reverse-proxy reverse-proxy.home 46.226.104.98
User geir
IdentityFile ~/.ssh/id_ed25519_dev
@ -66,12 +66,12 @@
IdentityFile ~/.ssh/id_ed25519_admin
Host admin-grey grey-area.admin
Hostname 10.0.0.11
Hostname 10.0.0.12
User sma
IdentityFile ~/.ssh/id_ed25519_admin
Host admin-reverse reverse-proxy.admin
Hostname 10.0.0.12
Hostname 46.226.104.98
User sma
IdentityFile ~/.ssh/id_ed25519_admin

View file

@ -7,6 +7,8 @@
users.users.sma = {
description = "Diziet Sma - System Administrator";
isNormalUser = true;
uid = 1001; # Fixed UID for consistency across machines
group = "sma"; # Primary group
# Admin privileges
extraGroups = [
@ -126,7 +128,12 @@
# Admin user home directory permissions
systemd.tmpfiles.rules = [
"d /home/sma 0755 sma users -"
"d /home/sma/.ssh 0700 sma users -"
"d /home/sma 0755 sma sma -"
"d /home/sma/.ssh 0700 sma sma -"
];
# Create the sma group
users.groups.sma = {
gid = 992; # Fixed GID for consistency across machines
};
}

View file

@ -10,10 +10,6 @@
incus
lxc
];
users.users.geir = {
extraGroups = [
"incus-admin"
];
};
networking.firewall.allowedTCPPorts = [ 8443 ];
}

473
plan.md
View file

@ -111,6 +111,8 @@ Home-lab/
- **SSH Infrastructure**: Implemented centralized SSH key management
- **Boot Performance**: Clean boot in ~1 minute with ZFS auto-mounting enabled
- **Remote Deployment**: Established rsync + SSH deployment workflow
- **NFS Server**: Configured NFS exports for both local (10.0.0.0/24) and Tailscale (100.64.0.0/10) networks
- **Network Configuration**: Updated to use Tailscale IPs for reliable mesh connectivity
#### Technical Solutions:
- **ZFS Native Mounting**: Migrated from legacy mountpoints to ZFS native paths
@ -118,26 +120,91 @@ Home-lab/
- **Graphics Compatibility**: Added `nomodeset` kernel parameter, disabled NVIDIA drivers
- **DNS Configuration**: Multi-tier DNS with Pi-hole primary, router and Google fallback
- **Deployment Method**: Remote deployment via rsync + SSH instead of direct nixos-rebuild
- **NFS Exports**: Resolved dataset conflicts by commenting out conflicting tmpfiles rules
- **Network Access**: Added Tailscale interface (tailscale0) as trusted interface in firewall
#### Data Verified:
- **Storage Pool**: 903GB used, 896GB available
- **Media Content**: Films (184GB), Series (612GB), Audiobooks (94GB), Music (9.1GB), Books (3.5GB)
- **Mount Points**: `/mnt/storage` and `/mnt/storage/media` with proper ZFS auto-mounting
- **NFS Access**: Both datasets exported with proper permissions for network access
#### Next Steps for sleeper-service:
- [ ] Implement automated backup services
- [ ] Add system monitoring and alerting
- [ ] Configure additional NFS exports as needed
- [ ] Plan storage expansion strategy
### grey-area Deployment (COMPLETED) ✅ NEW
**Date**: June 2025
**Status**: ✅ Fully operational
**Machine**: Intel Xeon E5-2670 v3 (24 cores) @ 3.10 GHz, 31.24 GiB RAM
#### Lessons Learned:
1. **ZFS Mounting Strategy**: Native ZFS mountpoints are more reliable than legacy mounts in NixOS
2. **Remote Deployment**: rsync + SSH approach avoids local machine conflicts during deployment
3. **DNS Configuration**: Manual DNS configuration crucial during initial deployment phase
4. **Graphics Compatibility**: `nomodeset` parameter essential for headless server deployment
5. **Boot Troubleshooting**: ZFS auto-mounting conflicts can be resolved by removing hardware-configuration.nix ZFS entries
6. **Data Migration**: ZFS dataset property changes can be done safely without data loss
7. **Network Integration**: Pi-hole DNS integration significantly improves package resolution reliability
#### Key Achievements:
- **Flake Configuration**: Successfully deployed NixOS flake-based configuration
- **NFS Client**: Configured reliable NFS mount to sleeper-service media storage via Tailscale
- **Service Stack**: Deployed comprehensive application server with multiple services
- **Network Integration**: Integrated with centralized extraHosts module using Tailscale IPs
- **User Management**: Resolved UID conflicts and implemented consistent user configuration
- **Firewall Configuration**: Properly configured ports for all services
#### Services Deployed:
- **Jellyfin**: ✅ Media server with access to NFS-mounted content from sleeper-service
- **Calibre-web**: ✅ E-book management and reading interface
- **Forgejo**: ✅ Git hosting server (git.geokkjer.eu) with reverse proxy integration
- **Audiobook Server**: ✅ Audiobook streaming and management
#### Technical Implementation:
- **NFS Mount**: `/mnt/remote/media` successfully mounting `sleeper-service:/mnt/storage/media`
- **Network Path**: Using Tailscale mesh (100.x.x.x) for reliable connectivity
- **Mount Options**: Configured with automount, soft mount, and appropriate timeouts
- **Firewall Ports**: 22 (SSH), 3000 (Forgejo), 23231 (other services)
- **User Configuration**: Fixed UID consistency with centralized sma user module
#### Data Access Verified:
- **Movies**: 38 films accessible via NFS
- **TV Series**: 29 series collections
- **Music**: 9 music directories
- **Audiobooks**: 79 audiobook collections
- **Books**: E-book collection
- **Media Services**: All content accessible through Jellyfin and other services
### reverse-proxy Integration (COMPLETED) ✅ NEW
**Date**: June 2025
**Status**: ✅ Fully operational
**Machine**: External VPS (46.226.104.98)
#### Key Achievements:
- **Nginx Configuration**: Successfully configured reverse proxy for Forgejo
- **Hostname Resolution**: Fixed hostname mapping from incorrect "apps" to correct "grey-area"
- **SSL/TLS**: Configured ACME Let's Encrypt certificate for git.geokkjer.eu
- **SSH Forwarding**: Configured SSH proxy on port 1337 for Git operations
- **Network Security**: Implemented DMZ-style security with Tailscale-only SSH access
#### Technical Configuration:
- **HTTP Proxy**: `git.geokkjer.eu``http://grey-area:3000` (Forgejo)
- **SSH Proxy**: Port 1337 → `grey-area:22` for Git SSH operations
- **Network Path**: External traffic → reverse-proxy → Tailscale → grey-area
- **Security**: SSH restricted to Tailscale network, fail2ban protection
- **DNS**: Proper hostname resolution via extraHosts module
### Centralized Network Configuration (COMPLETED) ✅ NEW
**Date**: June 2025
**Status**: ✅ Fully operational
#### Key Achievements:
- **extraHosts Module**: Created centralized hostname resolution using Tailscale IPs
- **Network Consistency**: All machines use same IP mappings for reliable mesh connectivity
- **SSH Configuration**: Updated IP addresses in ssh-keys.nix module
- **User Management**: Resolved user configuration conflicts between modules
#### Network Topology:
- **Tailscale Mesh IPs**:
- `100.109.28.53` - congenital-optimist (workstation)
- `100.81.15.84` - sleeper-service (NFS file server)
- `100.119.86.92` - grey-area (application server)
- `100.96.189.104` - reverse-proxy (external VPS)
- `100.103.143.108` - pihole (DNS server)
- `100.126.202.40` - wordpresserver (legacy)
#### Module Integration:
- **extraHosts**: Added to all machine configurations for consistent hostname resolution
- **SSH Keys**: Updated IP addresses (grey-area: 10.0.0.12, reverse-proxy: 46.226.104.98)
- **User Modules**: Fixed conflicts between sma user definitions in different modules
### Home Lab Deployment Tool (COMPLETED) ✅ NEW
**Date**: Recently completed
@ -408,29 +475,79 @@ Home-lab/
- [ ] Verify shell environment and modern CLI tools work
- [ ] Test console theming and TTY setup
## Phase 4: Literate Dotfiles Setup
## Phase 4: Dotfiles & Configuration Management
### 4.1 Per-User Org-mode Infrastructure
- [ ] Create per-user dotfiles directories (`users/geir/dotfiles/`)
- [ ] Create comprehensive `users/geir/dotfiles/README.org` with auto-tangling
- [ ] Set up Emacs configuration for literate programming workflow
- [ ] Configure automatic tangling on save
- [ ] Create modular sections for different tool configurations
- [ ] Plan for additional users (admin, service accounts, etc.)
### 4.1 GNU Stow Infrastructure for Regular Dotfiles ✅ DECIDED
**Approach**: Use GNU Stow for traditional dotfiles, literate programming for Emacs only
### 4.2 Configuration Domains
- [ ] Shell configuration (zsh, starship, aliases)
- [ ] Editor configurations (emacs, neovim, vscode)
- [ ] Development tools and environments
- [ ] System-specific tweaks and preferences
- [ ] Git configuration and development workflow
#### GNU Stow Setup
- [ ] Create `~/dotfiles/` directory structure with package-based organization
- [ ] Set up core packages: `zsh/`, `git/`, `tmux/`, `starship/`, etc.
- [ ] Configure selective deployment per machine (workstation vs servers)
- [ ] Create stow deployment scripts for different machine profiles
- [ ] Document stow workflow and package management
### 4.3 Integration with NixOS
- [ ] Link org-mode generated configs with NixOS modules where appropriate
- [ ] Document the relationship between system-level and user-level configs
- [ ] Create per-user configuration templates for common patterns
- [ ] Plan user-specific configurations vs shared configurations
- [ ] Consider user isolation and security implications
#### Package Structure
```
~/dotfiles/ # Stow directory (target: $HOME)
├── zsh/ # Shell configuration
│ ├── .zshrc
│ ├── .zshenv
│ └── .config/zsh/
├── git/ # Git configuration
│ ├── .gitconfig
│ └── .config/git/
├── starship/ # Prompt configuration
│ └── .config/starship.toml
├── tmux/ # Terminal multiplexer
│ └── .tmux.conf
├── emacs/ # Basic Emacs bootstrap (points to literate config)
│ └── .emacs.d/early-init.el
└── machine-specific/ # Per-machine configurations
├── workstation/
└── server/
```
### 4.2 Literate Programming for Emacs Configuration ✅ DECIDED
**Approach**: Comprehensive org-mode literate configuration for Emacs only
#### Emacs Literate Setup
- [ ] Create `~/dotfiles/emacs/.emacs.d/configuration.org` as master config
- [ ] Set up automatic tangling on save (org-babel-tangle-on-save)
- [ ] Modular org sections: packages, themes, keybindings, workflows
- [ ] Bootstrap early-init.el to load tangled configuration
- [ ] Create machine-specific customizations within org structure
#### Literate Configuration Structure
```
~/dotfiles/emacs/.emacs.d/
├── early-init.el # Bootstrap (generated by Stow)
├── configuration.org # Master literate config
├── init.el # Tangled from configuration.org
├── modules/ # Tangled module files
│ ├── base.el
│ ├── development.el
│ ├── org-mode.el
│ └── ui.el
└── machine-config/ # Machine-specific overrides
├── workstation.el
└── server.el
```
### 4.3 Integration Strategy
- [ ] **System-level**: NixOS modules provide system packages and environment
- [ ] **User-level**: GNU Stow manages dotfiles and application configurations
- [ ] **Emacs-specific**: Org-mode literate programming for comprehensive Emacs setup
- [ ] **Per-machine**: Selective stow packages + machine-specific customizations
- [ ] **Version control**: Git repository for dotfiles with separate org documentation
### 4.4 Deployment Workflow
- [ ] Create deployment scripts for different machine types:
- **Workstation**: Full package deployment (zsh, git, tmux, starship, emacs)
- **Server**: Minimal package deployment (zsh, git, basic emacs)
- **Development**: Additional packages (language-specific tools, IDE configs)
- [ ] Integration with existing `lab` deployment tool
- [ ] Documentation for new user onboarding across machines
## Phase 5: Home Lab Expansion Planning
@ -451,20 +568,27 @@ Home-lab/
- [x] Network configuration with Pi-hole DNS integration
- [x] System boots cleanly in ~1 minute with ZFS auto-mounting
- [x] Data preservation verified (Films: 184GB, Series: 612GB, etc.)
- [x] NFS exports configured for both local and Tailscale networks
- [x] Resolved dataset conflicts and tmpfiles rule conflicts
- [ ] Automated backup services (future enhancement)
- [ ] System monitoring and alerting (future enhancement)
- [ ] **reverse-proxy** edge server:
- Nginx/Traefik/caddy reverse proxy
- SSL/TLS termination with Let's Encrypt
- External access gateway and load balancing
- Security protection (Fail2ban, rate limiting)
- Minimal attack surface, headless operation
- [ ] **grey-area** application server (Culture GCU - versatile, multi-purpose):
- **Primary**: Forgejo Git hosting (repositories, CI/CD, project management)
- **Secondary**: Jellyfin media server
- **Monitoring**: TBD
- **Infrastructure**: Container-focused (Podman), PostgreSQL database
- **Integration**: Central Git hosting for all home lab projects
- [x] **reverse-proxy** edge server: ✅ **COMPLETED**
- [x] Nginx reverse proxy with proper hostname mapping (grey-area vs apps)
- [x] SSL/TLS termination with Let's Encrypt for git.geokkjer.eu
- [x] External access gateway with DMZ security configuration
- [x] SSH forwarding on port 1337 for Git operations
- [x] Fail2ban protection and Tailscale-only SSH access
- [x] Minimal attack surface, headless operation
- [x] **grey-area** application server (Culture GCU - versatile, multi-purpose): ✅ **COMPLETED**
- [x] **Primary**: Forgejo Git hosting (git.geokkjer.eu) with reverse proxy integration
- [x] **Secondary**: Jellyfin media server with NFS-mounted content
- [x] **Additional**: Calibre-web e-book server and audiobook streaming
- [x] **Infrastructure**: Container-focused (Podman), NFS client for media storage
- [x] **Integration**: Central Git hosting accessible externally via reverse proxy
- [x] **Network**: Integrated with Tailscale mesh and centralized hostname resolution
- [x] **User Management**: Resolved UID conflicts with centralized sma user configuration
- [ ] **Monitoring**: TBD (future enhancement)
- [ ] **PostgreSQL**: Plan database services for applications requiring persistent storage
- [ ] Plan for additional users across machines:
- [x] **geir** - Primary user (development, desktop, daily use)
- [x] **sma** - Admin user (Diziet Sma, system administration, security oversight)
@ -516,18 +640,63 @@ Home-lab/
- [ ] Deployment automation
- [ ] Monitoring and alerting
### 6.3 Advanced Deployment Strategies
- [ ] **Research deploy-rs**: Investigate deploy-rs as alternative to custom lab script
- Evaluate Rust-based deployment tool for NixOS flakes
- Compare features: parallel deployment, rollback capabilities, health checks
- Assess integration with existing SSH key management and Tailscale network
- Consider migration path from current rsync + SSH approach
- [ ] **Convert lab script to Guile Scheme**: Explore functional deployment scripting
- Research Guile Scheme for system administration scripting
- Evaluate benefits: better error handling, functional composition, extensibility
- Design modular deployment pipeline with Scheme
- Consider integration with GNU Guix deployment patterns
- Plan migration strategy from current shell script implementation
### 6.3 Advanced Deployment Strategies ✅ RESEARCH COMPLETED
#### Deploy-rs Migration (Priority: High) 📋 RESEARCHED
- [x] **Research deploy-rs capabilities** ✅ COMPLETED
- [x] Rust-based deployment tool specifically designed for NixOS flakes
- [x] Features: parallel deployment, automatic rollback, health checks, SSH-based
- [x] Advanced capabilities: atomic deployments, magic rollback on failure
- [x] Profile management: system, user, and custom profiles support
- [x] Integration potential: Works with existing SSH keys and Tailscale network
- [ ] **Migration Planning**: Transition from custom `lab` script to deploy-rs
- [ ] Create deploy-rs configuration in flake.nix for all 4 machines
- [ ] Configure nodes: sleeper-service, grey-area, reverse-proxy, congenital-optimist
- [ ] Set up health checks for critical services (NFS, Forgejo, Jellyfin, nginx)
- [ ] Test parallel deployment capabilities across infrastructure
- [ ] Implement automatic rollback for failed deployments
- [ ] Document migration benefits and new deployment workflow
#### Deploy-rs Configuration Structure
```nix
# flake.nix additions
deploy.nodes = {
sleeper-service = {
hostname = "100.81.15.84"; # Tailscale IP
profiles.system.path = deploy-rs.lib.x86_64-linux.activate.nixos
self.nixosConfigurations.sleeper-service;
profiles.system.user = "root";
};
grey-area = {
hostname = "100.119.86.92";
profiles.system.path = deploy-rs.lib.x86_64-linux.activate.nixos
self.nixosConfigurations.grey-area;
# Health checks for Forgejo, Jellyfin services
};
reverse-proxy = {
hostname = "100.96.189.104";
profiles.system.path = deploy-rs.lib.x86_64-linux.activate.nixos
self.nixosConfigurations.reverse-proxy;
# Health checks for nginx, SSL certificates
};
};
```
#### Migration Benefits
- **Atomic deployments**: Complete success or automatic rollback
- **Parallel deployment**: Deploy to multiple machines simultaneously
- **Health checks**: Validate services after deployment
- **Connection resilience**: Better handling of SSH/network issues
- **Flake-native**: Designed specifically for NixOS flake workflows
- **Safety**: Magic rollback prevents broken deployments
#### Alternative: Guile Scheme Exploration (Priority: Low)
- [ ] **Research Guile Scheme for system administration**
- [ ] Evaluate functional deployment scripting patterns
- [ ] Compare with current shell script and deploy-rs approaches
- [ ] Consider integration with GNU Guix deployment patterns
- [ ] Assess learning curve vs. practical benefits for home lab use case
### 6.4 Writeup
- [ ] Take all the knowledge we have amassed and make a blog post or a series of blog posts
@ -560,20 +729,114 @@ Home-lab/
- Document manual recovery procedures
- Preserve current user configuration during migration
## Current Status Overview (Updated December 2024)
### Infrastructure Deployment Status ✅ MAJOR MILESTONE ACHIEVED
**PHASE 1**: Flakes Migration - **COMPLETED**
**PHASE 2**: Configuration Cleanup - **COMPLETED**
**PHASE 3**: System Upgrade & Validation - **COMPLETED**
**PHASE 5**: Home Lab Expansion - **4/4 MACHINES FULLY OPERATIONAL** 🎉
### Machine Status
- ✅ **congenital-optimist**: Development workstation (fully operational)
- ✅ **sleeper-service**: NFS file server with 903GB media library (fully operational)
- ✅ **grey-area**: Application server with Forgejo, Jellyfin, Calibre-web, audiobook server (fully operational)
- ✅ **reverse-proxy**: External gateway with nginx, SSL termination, SSH forwarding (fully operational)
### Network Architecture Status
- ✅ **Tailscale Mesh**: All machines connected via secure mesh network (100.x.x.x addresses)
- ✅ **Hostname Resolution**: Centralized extraHosts module deployed across all machines
- ✅ **NFS Storage**: Reliable media storage access via Tailscale network (sleeper-service → grey-area)
- ✅ **External Access**: Public services accessible via git.geokkjer.eu with SSL
- ✅ **SSH Infrastructure**: Centralized key management with role-based access patterns
- ✅ **Firewall Configuration**: Service ports properly configured across all machines
### Services Status - FULLY OPERATIONAL STACK 🚀
- ✅ **Git Hosting**: Forgejo operational at git.geokkjer.eu with SSH access on port 1337
- ✅ **Media Streaming**: Jellyfin with NFS-mounted content library (38 movies, 29 TV series)
- ✅ **E-book Management**: Calibre-web for book collections
- ✅ **Audiobook Streaming**: Audiobook server with 79 audiobook collections
- ✅ **File Storage**: NFS server with 903GB media library accessible across network
- ✅ **Web Gateway**: Nginx reverse proxy with Let's Encrypt SSL and proper hostname mapping
- ✅ **User Management**: Consistent UID/GID configuration across machines (sma user: 1001/992)
### Infrastructure Achievements - COMPREHENSIVE DEPLOYMENT ✅
- ✅ **NFS Mount Resolution**: Fixed grey-area `/mnt/storage``/mnt/storage/media` dataset access
- ✅ **Network Exports**: Updated sleeper-service NFS exports for Tailscale network (100.64.0.0/10)
- ✅ **Service Discovery**: Corrected reverse-proxy hostname mapping from "apps" to "grey-area"
- ✅ **Firewall Management**: Added port 3000 for Forgejo service accessibility
- ✅ **SSH Forwarding**: Configured SSH proxy on port 1337 for Git operations
- ✅ **SSL Termination**: Let's Encrypt certificates working for git.geokkjer.eu
- ✅ **Data Verification**: All media content accessible (movies, TV, music, audiobooks, books)
- ✅ **Deployment Tools**: Custom `lab` command operational for infrastructure management
### Current Operational Status
**🟢 ALL CORE INFRASTRUCTURE DEPLOYED AND OPERATIONAL**
- **4/4 machines deployed** with full service stack
- **External access verified**: `curl -I https://git.geokkjer.eu` returns HTTP/2 200
- **NFS connectivity confirmed**: Media files accessible across network via Tailscale
- **Service integration complete**: Forgejo, Jellyfin, Calibre-web, audiobook server running
- **Network mesh stable**: All machines connected via Tailscale with centralized hostname resolution
### Next Phase Priorities
- [ ] **PHASE 4**: GNU Stow + Literate Emacs Setup
- [ ] Set up GNU Stow infrastructure for regular dotfiles (zsh, git, tmux, starship)
- [ ] Create comprehensive Emacs literate configuration with org-mode
- [ ] Implement selective deployment per machine type (workstation vs server)
- [ ] Integration with existing NixOS system-level configuration
- [ ] **PHASE 6**: Advanced Features & Deploy-rs Migration
- [ ] Migrate from custom `lab` script to deploy-rs for improved deployment
- [ ] Implement system monitoring and alerting infrastructure
- [ ] Set up automated backup services for critical data
- [ ] Create health checks and deployment validation
- [ ] **Documentation & Knowledge Sharing**
- [ ] Comprehensive blog post series documenting the full home lab journey
- [ ] User guides for GNU Stow + literate Emacs configuration workflow
- [ ] Deploy-rs migration guide and lessons learned
- [ ] **Future Enhancements**
- [ ] User ID consistency cleanup (sma user UID alignment across machines)
- [ ] CI/CD integration with Forgejo for automated testing and deployment
---
## Success Criteria
- [ ] System boots reliably with flake configuration
- [ ] All current functionality preserved
- [ ] NixOS 25.05 running stable
- [ ] Configuration is modular and maintainable
- [ ] User environment fully functional with all packages
- [ ] Modern CLI tools and aliases working
- [ ] Console theming preserved
- [ ] Virtualization stack operational
- [ ] Literate dotfiles workflow established
- [ ] Ready for multi-machine expansion
- [ ] Development workflow improved
- [ ] Documentation complete for future reference
### Core Infrastructure ✅ FULLY ACHIEVED 🎉
- [x] System boots reliably with flake configuration
- [x] All current functionality preserved
- [x] NixOS 25.05 running stable across all machines
- [x] Configuration is modular and maintainable
- [x] User environment fully functional with all packages
- [x] Modern CLI tools and aliases working
- [x] Console theming preserved
- [x] Virtualization stack operational
- [x] **Multi-machine expansion completed (4/4 machines deployed)**
- [x] Development workflow improved with Git hosting
### Service Architecture ✅ FULLY ACHIEVED 🚀
- [x] NFS file server operational with reliable network access via Tailscale
- [x] Git hosting with external access via reverse proxy (git.geokkjer.eu)
- [x] Media services with shared storage backend (Jellyfin + 903GB library)
- [x] E-book and audiobook management services operational
- [x] Secure external access with SSL termination and SSH forwarding
- [x] Network mesh connectivity with centralized hostname resolution
- [x] **All services verified operational and accessible externally**
### Network Integration ✅ FULLY ACHIEVED 🌐
- [x] Tailscale mesh network connecting all infrastructure machines
- [x] Centralized hostname resolution via extraHosts module
- [x] NFS file sharing working reliably over network
- [x] SSH key management with role-based access patterns
- [x] Firewall configuration properly securing all services
- [x] **External domain (git.geokkjer.eu) with SSL certificates working**
### Outstanding Enhancement Goals 🔄
- [ ] Literate dotfiles workflow established with org-mode
- [ ] Documentation complete for future reference and blog writeup
- [ ] System monitoring and alerting infrastructure (Prometheus/Grafana)
- [ ] Automated deployment and maintenance improvements
- [ ] Automated backup services for critical data
- [ ] User ID consistency cleanup across machines
## Infrastructure Notes
@ -610,10 +873,10 @@ Home-lab/
- **Hardware**: Intel Xeon E5-2670 v3 (24 cores) @ 3.10 GHz, 31.24 GiB RAM
- **Primary Mission**: Forgejo Git hosting and project management
- **Performance**: Excellent specs for heavy containerized workloads and CI/CD
- Container-focused architecture using Podman
- PostgreSQL database for Forgejo
- Concurrent multi-service deployment capability
- Secondary services: Jellyfin (with transcoding), Nextcloud, Grafana
- **Container-focused architecture** using Podman
- **PostgreSQL database** for Forgejo
- **Concurrent multi-service deployment capability**
- **Secondary services**: Jellyfin (with transcoding), Nextcloud, Grafana
- Integration hub for all home lab development projects
- Culture name fits: "versatile ship handling varied, ambiguous tasks"
- Central point for CI/CD pipelines and automation
@ -624,3 +887,67 @@ Home-lab/
- Modular NixOS configuration allows easy machine additions
- Per-user dotfiles structure scales across multiple machines
- Tailscale provides secure network foundation for multi-machine setup
#### Recent Critical Issue Resolution (December 2024) 🔧
**NFS Mount and Service Integration Issues - RESOLVED**
1. **NFS Dataset Structure Resolution**:
- **Problem**: grey-area couldn't access media files via NFS mount
- **Root Cause**: ZFS dataset structure confusion - mounting `/mnt/storage` vs `/mnt/storage/media`
- **Solution**: Updated grey-area NFS mount from `sleeper-service:/mnt/storage` to `sleeper-service:/mnt/storage/media`
- **Result**: All media content now accessible (38 movies, 29 TV series, 9 music albums, 79 audiobooks)
2. **NFS Network Export Configuration**:
- **Problem**: NFS exports only configured for local network (10.0.0.0/24)
- **Root Cause**: Missing Tailscale network access in NFS exports
- **Solution**: Updated sleeper-service NFS exports to include Tailscale network (100.64.0.0/10)
- **Result**: Reliable NFS connectivity over Tailscale mesh network
3. **Conflicting tmpfiles Rules**:
- **Problem**: systemd tmpfiles creating conflicting directory structures for NFS exports
- **Root Cause**: tmpfiles.d rules interfering with ZFS dataset mounting
- **Solution**: Commented out conflicting tmpfiles rules in sleeper-service configuration
- **Result**: Clean NFS export structure without mounting conflicts
4. **Forgejo Service Accessibility**:
- **Problem**: git.geokkjer.eu returning connection refused errors
- **Root Cause**: Multiple issues - firewall ports, hostname mapping, SSH forwarding
- **Solutions Applied**:
- Added port 3000 to grey-area firewall configuration
- Fixed reverse-proxy nginx configuration: `http://apps:3000``http://grey-area:3000`
- Updated SSH forwarding: `apps:22``grey-area:22` for port 1337
- **Result**: External access verified - `curl -I https://git.geokkjer.eu` returns HTTP/2 200
5. **Hostname Resolution Consistency**:
- **Problem**: Inconsistent hostname references across configurations ("apps" vs "grey-area")
- **Root Cause**: Legacy hostname references in reverse-proxy configuration
- **Solution**: Updated all configurations to use consistent "grey-area" hostname
- **Result**: Proper service discovery and reverse proxy routing
6. **User ID Consistency Challenge**:
- **Current State**: sma user has UID 1003 on grey-area vs 1001 on sleeper-service
- **Workaround**: NFS access working via group permissions (users group: GID 100)
- **Future Fix**: Implement centralized UID management across all machines
#### Recent Troubleshooting & Solutions (June 2025):
8. **NFS Dataset Structure**: Proper understanding of ZFS dataset hierarchy crucial for NFS exports
- `/mnt/storage` vs `/mnt/storage/media` dataset mounting differences
- NFS exports must match actual ZFS dataset structure, not subdirectories
- Client mount paths must align with server export paths for data access
9. **Network Transition Management**: Tailscale vs local network connectivity during deployment
- NFS exports need both local (10.0.0.0/24) and Tailscale (100.64.0.0/10) network access
- extraHosts module provides consistent hostname resolution across network changes
- Firewall configuration must accommodate service ports for external access
10. **Reverse Proxy Configuration**: Hostname consistency critical for proxy functionality
- nginx upstream configuration must use correct hostnames (grey-area not apps)
- Service discovery relies on centralized hostname resolution modules
- SSL certificate management works seamlessly with proper nginx configuration
11. **Service Integration**: Multi-machine service architecture requires coordinated configuration
- Forgejo deployment spans grey-area (service) + reverse-proxy (gateway) + DNS (domain)
- NFS client/server coordination requires matching export/mount configurations
- User ID consistency across machines essential for NFS file access permissions
12. **Firewall Management**: Service-specific port configuration essential for functionality
- Application servers need service ports opened (3000 for Forgejo, etc.)
- Reverse proxy needs external ports (80, 443, 1337) and internal connectivity
- SSH access coordination between local and Tailscale networks for security

467
research/forgejo.md Normal file
View file

@ -0,0 +1,467 @@
# Forgejo Best Practices and Configuration Guide
## Overview
Forgejo is a self-hosted lightweight software forge focused on scaling, federation, and privacy. This document outlines best practices for configuring and securing a Forgejo instance, along with potential improvements for the current setup.
## Current Configuration Analysis
### Current Setup Summary
- **Service**: Forgejo running on grey-area machine
- **User**: `git` user (follows best practices)
- **Mode**: Production (`RUN_MODE = "prod"`)
- **URL**: `https://git.geokkjer.eu`
- **SSH**: Port 1337 on `git.geokkjer.eu`
- **Registration**: Disabled (security best practice)
### Configuration Location
- Configuration file: `/home/geir/Home-lab/machines/grey-area/services/forgejo.nix`
## Security Best Practices
### 1. Authentication and Access Control
**Current Status**: ✅ Registration disabled
```nix
service = {
DISABLE_REGISTRATION = true;
};
```
**Recommendations**:
- Consider implementing OAuth2/OIDC for centralized authentication
- Enable two-factor authentication (2FA)
- Set up proper user management policies
**Potential Improvements**:
```nix
service = {
DISABLE_REGISTRATION = true;
REQUIRE_SIGNIN_VIEW = true; # Require login to view repos
ENABLE_CAPTCHA = true; # Enable captcha for forms
DEFAULT_ALLOW_CREATE_ORGANIZATION = false;
};
security = {
INSTALL_LOCK = true;
SECRET_KEY = "your-secret-key-here"; # Use secrets management
LOGIN_REMEMBER_DAYS = 7;
COOKIE_REMEMBER_NAME = "forgejo_incredible";
COOKIE_USERNAME = "forgejo_username";
COOKIE_SECURE = true; # HTTPS only cookies
ENABLE_LOGIN_STATUS_COOKIE = true;
};
```
### 2. SSH Configuration
**Current Status**: ✅ Custom SSH port (1337)
```nix
server = {
SSH_PORT = 1337;
};
```
**Additional SSH Security**:
```nix
ssh = {
DISABLE_SSH = false;
START_SSH_SERVER = true;
SSH_SERVER_HOST_KEYS = "ssh/forgejo.rsa, ssh/gogs.rsa";
SSH_KEY_TEST_PATH = "/tmp";
SSH_KEYGEN_PATH = "ssh-keygen";
SSH_SERVER_CIPHERS = "chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr";
SSH_SERVER_KEY_EXCHANGES = "curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1";
SSH_SERVER_MACS = "hmac-sha2-256-etm@openssh.com,hmac-sha2-256,hmac-sha1";
};
```
### 3. Database Security
**Recommendation**: Use PostgreSQL instead of SQLite for production
```nix
database = {
DB_TYPE = "postgres";
HOST = "127.0.0.1:5432";
NAME = "forgejo";
USER = "forgejo";
PASSWD = "secure-password"; # Use secrets management
SSL_MODE = "require";
CHARSET = "utf8";
};
```
## Performance Optimization
### 1. Caching Configuration
```nix
cache = {
ADAPTER = "redis";
INTERVAL = 60;
HOST = "127.0.0.1:6379";
ITEM_TTL = "16h";
};
session = {
PROVIDER = "redis";
PROVIDER_CONFIG = "127.0.0.1:6379";
COOKIE_NAME = "i_like_forgejo";
COOKIE_SECURE = true;
GC_INTERVAL_TIME = 86400;
SESSION_LIFE_TIME = 86400;
DOMAIN = "git.geokkjer.eu";
};
```
### 2. Repository Management
```nix
repository = {
ENABLE_PUSH_CREATE_USER = true; # Already configured ✅
ENABLE_PUSH_CREATE_ORG = false;
DEFAULT_BRANCH = "main";
PREFERRED_LICENSES = "Apache License 2.0,MIT License,GPL-3.0-or-later";
DISABLE_HTTP_GIT = false;
ACCESS_CONTROL_ALLOW_ORIGIN = "";
USE_COMPAT_SSH_URI = false;
DEFAULT_CLOSE_ISSUES_VIA_COMMITS_IN_ANY_BRANCH = false;
ENABLE_PUSH_CREATE_USER = true;
ENABLE_PUSH_CREATE_ORG = false;
};
```
### 3. Indexer Configuration
```nix
indexer = {
ISSUE_INDEXER_TYPE = "bleve";
ISSUE_INDEXER_PATH = "indexers/issues.bleve";
REPO_INDEXER_ENABLED = true;
REPO_INDEXER_PATH = "indexers/repos.bleve";
UPDATE_BUFFER_LEN = 20;
MAX_FILE_SIZE = 1048576;
};
```
## Backup and Disaster Recovery
### 1. Data Backup Strategy
```bash
# Database backup (if using PostgreSQL)
pg_dump -U forgejo -h localhost forgejo > forgejo_backup_$(date +%Y%m%d).sql
# Repository data backup
tar -czf forgejo_repos_$(date +%Y%m%d).tar.gz /var/lib/forgejo/data/
# Configuration backup
cp /etc/forgejo/app.ini forgejo_config_$(date +%Y%m%d).ini
```
### 2. Automated Backup with NixOS
```nix
# Add to configuration.nix
services.postgresqlBackup = {
enable = true;
databases = [ "forgejo" ];
startAt = "daily";
location = "/backup/postgresql";
};
systemd.services.forgejo-backup = {
description = "Backup Forgejo repositories";
startAt = "daily";
script = ''
${pkgs.gnutar}/bin/tar -czf /backup/forgejo/repos_$(date +%Y%m%d).tar.gz /var/lib/forgejo/data/
'';
serviceConfig = {
Type = "oneshot";
User = "root";
};
};
```
## Monitoring and Logging
### 1. Logging Configuration
```nix
log = {
MODE = "file";
LEVEL = "Info";
ROOT_PATH = "/var/log/forgejo";
REDIRECT_MACARON_LOG = true;
MACARON_PREFIX = "[Macaron]";
ROUTER_LOG_LEVEL = "Info";
ROUTER_PREFIX = "[Router]";
ENABLE_SSH_LOG = true;
};
```
### 2. Metrics and Monitoring
```nix
metrics = {
ENABLED = true;
TOKEN = "your-metrics-token"; # Use secrets management
};
# Add Prometheus monitoring
services.prometheus.exporters.forgejo = {
enable = true;
port = 3001;
configFile = "/etc/forgejo/app.ini";
};
```
## Email Configuration
### SMTP Setup
```nix
mailer = {
ENABLED = true;
SMTP_ADDR = "smtp.your-provider.com";
SMTP_PORT = 587;
FROM = "noreply@geokkjer.eu";
USER = "your-smtp-user";
PASSWD = "your-smtp-password"; # Use secrets management
ENABLE_HELO = true;
HELO_HOSTNAME = "git.geokkjer.eu";
SKIP_VERIFY = false;
USE_CERTIFICATE = true;
CERT_FILE = "/path/to/cert.pem";
KEY_FILE = "/path/to/key.pem";
IS_TLS_ENABLED = true;
};
```
## Web UI and UX Improvements
### 1. UI Configuration
```nix
ui = {
EXPLORE_PAGING_NUM = 20;
ISSUE_PAGING_NUM = 20;
MEMBERS_PAGING_NUM = 20;
FEED_MAX_COMMIT_NUM = 5;
FEED_PAGING_NUM = 20;
SITEMAP_PAGING_NUM = 20;
GRAPH_MAX_COMMIT_NUM = 100;
CODE_COMMENT_LINES = 4;
DEFAULT_SHOW_FULL_NAME = false;
SEARCH_REPO_DESCRIPTION = true;
USE_SERVICE_WORKER = true;
};
"ui.meta" = {
AUTHOR = "Forgejo";
DESCRIPTION = "Git with a cup of tea! Painless self-hosted git service.";
KEYWORDS = "go,git,self-hosted,gitea,forgejo";
};
```
### 2. Webhook Configuration
```nix
webhook = {
QUEUE_LENGTH = 1000;
DELIVER_TIMEOUT = 5;
SKIP_TLS_VERIFY = false;
ALLOWED_HOST_LIST = "";
PAGING_NUM = 10;
PROXY_URL = "";
PROXY_HOSTS = "";
};
```
## Federation (Future Feature)
Forgejo is working on ActivityPub federation support:
```nix
federation = {
ENABLED = false; # Not yet available
SHARE_USER_STATISTICS = false;
MAX_SIZE = 4;
ALGORITHMS = "rsa-sha256,rsa-sha512,ed25519";
};
```
## Potential Improvements for Current Setup
### 1. Immediate Improvements
1. **Add Database Configuration**:
- Migrate from SQLite to PostgreSQL for better performance
- Configure connection pooling
2. **Enhance Security**:
- Add `REQUIRE_SIGNIN_VIEW = true` to require authentication for viewing
- Configure proper SSL/TLS settings
- Implement secrets management for sensitive values
3. **Add Monitoring**:
- Enable metrics collection
- Set up log rotation
- Configure health checks
4. **Backup Strategy**:
- Implement automated backups
- Set up off-site backup storage
### 2. Medium-term Improvements
1. **Performance Optimization**:
- Add Redis for caching and sessions
- Configure repository indexing
- Optimize garbage collection
2. **User Experience**:
- Configure email notifications
- Set up custom themes/branding
- Add webhook integrations
3. **Integration**:
- Set up CI/CD integration
- Configure external authentication (LDAP/OAuth)
- Add container registry support
### 3. Enhanced Configuration Example
```nix
{ pkgs, config, ... }:
{
services.forgejo = {
enable = true;
user = "git";
stateDir = "/var/lib/forgejo";
database = {
type = "postgres";
host = "127.0.0.1";
port = 5432;
name = "forgejo";
user = "forgejo";
passwordFile = "/run/secrets/forgejo-db-password";
};
};
services.forgejo.settings = {
DEFAULT = {
RUN_MODE = "prod";
WORK_PATH = "/var/lib/forgejo";
};
service = {
DISABLE_REGISTRATION = true;
REQUIRE_SIGNIN_VIEW = true;
ENABLE_CAPTCHA = true;
DEFAULT_ALLOW_CREATE_ORGANIZATION = false;
};
server = {
ROOT_URL = "https://git.geokkjer.eu";
SSH_DOMAIN = "git.geokkjer.eu";
SSH_PORT = 1337;
CERT_FILE = "/etc/ssl/certs/forgejo.crt";
KEY_FILE = "/etc/ssl/private/forgejo.key";
DISABLE_SSH = false;
START_SSH_SERVER = true;
};
repository = {
ENABLE_PUSH_CREATE_USER = true;
ENABLE_PUSH_CREATE_ORG = false;
DEFAULT_BRANCH = "main";
PREFERRED_LICENSES = "Apache License 2.0,MIT License,GPL-3.0-or-later";
};
security = {
INSTALL_LOCK = true;
SECRET_KEY_PATH = "/run/secrets/forgejo-secret-key";
LOGIN_REMEMBER_DAYS = 7;
COOKIE_SECURE = true;
ENABLE_LOGIN_STATUS_COOKIE = true;
};
log = {
MODE = "file";
LEVEL = "Info";
ROOT_PATH = "/var/log/forgejo";
REDIRECT_MACARON_LOG = true;
};
metrics = {
ENABLED = true;
TOKEN_PATH = "/run/secrets/forgejo-metrics-token";
};
mailer = {
ENABLED = true;
SMTP_ADDR = "smtp.fastmail.com";
SMTP_PORT = 587;
FROM = "noreply@geokkjer.eu";
USER = "forgejo@geokkjer.eu";
PASSWD_PATH = "/run/secrets/forgejo-smtp-password";
IS_TLS_ENABLED = true;
};
other = {
SHOW_FOOTER_VERSION = true;
SHOW_FOOTER_TEMPLATE_LOAD_TIME = false;
};
};
# PostgreSQL configuration
services.postgresql = {
enable = true;
ensureDatabases = [ "forgejo" ];
ensureUsers = [
{
name = "forgejo";
ensurePermissions = {
"DATABASE forgejo" = "ALL PRIVILEGES";
};
}
];
};
# Redis for caching
services.redis.servers.forgejo = {
enable = true;
port = 6379;
bind = "127.0.0.1";
};
# Backup configuration
services.postgresqlBackup = {
enable = true;
databases = [ "forgejo" ];
startAt = "daily";
location = "/backup/postgresql";
};
# Secrets management
age.secrets = {
forgejo-db-password.file = ../secrets/forgejo-db-password.age;
forgejo-secret-key.file = ../secrets/forgejo-secret-key.age;
forgejo-metrics-token.file = ../secrets/forgejo-metrics-token.age;
forgejo-smtp-password.file = ../secrets/forgejo-smtp-password.age;
};
}
```
## Security Checklist
- [ ] **Authentication**: Disable registration, enable 2FA
- [ ] **Authorization**: Implement proper access controls
- [ ] **Encryption**: Use HTTPS, secure SSH configuration
- [ ] **Database**: Use PostgreSQL with SSL, regular backups
- [ ] **Secrets**: Use proper secrets management (agenix/sops)
- [ ] **Monitoring**: Enable logging, metrics collection
- [ ] **Updates**: Regular security updates, vulnerability scanning
- [ ] **Network**: Firewall configuration, rate limiting
- [ ] **Backup**: Automated backups, disaster recovery plan
## References
- [Forgejo Official Documentation](https://forgejo.org/docs/)
- [Forgejo Configuration Reference](https://forgejo.org/docs/latest/admin/config-cheat-sheet/)
- [Forgejo Security Guide](https://forgejo.org/docs/latest/admin/security/)
- [NixOS Forgejo Module](https://search.nixos.org/options?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=forgejo)
## Last Updated
June 7, 2025

View file

@ -248,7 +248,7 @@ Host git.geokkjer.eu
**Configuration**:
1. Enable nginx stream module
2. Configure Git SSH on port 2222
2. Configure Git SSH on port 1337
3. Update Forgejo SSH_DOMAIN setting
4. Test with alternative port