Compare commits

..

No commits in common. "a35d9ff4206c699a9a426a248326cb05a576036b" and "e976b14d19c4cbedcc61eae7609900e89a98f989" have entirely different histories.

26 changed files with 971 additions and 1158 deletions

View file

@ -3,42 +3,33 @@
## Overview
This part of the document provides general instructions for tha AI agent.
## General Instructions
- Treat this as iterative collaboration between user and AI agent
- **Context7 MCP is mandatory** for all technical documentation queries
- Use casual but knowledgeable tone - hobby/passion project, not corporate
- Update documentation frequently as project evolves
## General instructions
This document is to be treated as an iterative work and a collaberation between the user and AI agent.
## Language & Tool Preferences
- **Functional style preferred**
- **Guile Scheme**: System administration scripting (replace bash when possible)
- **Python**: AI tasks, when Python ecosystem is superior
- **TypeScript/JavaScript**: Web development (prefer Vite for new projects)
- **Rust**: Binary tools, performance-critical applications
- **Bash**: Only for very short scripts (< 10 lines)
## Goal for this file
Top part reusable instructions that can be transferred to other project and as away to iterativey make ai agent behave more like I prefer.
Bottom part should have information specific to the project.
The Plan.md file should have the project information and steps.
## Context7 MCP Usage Protocol
1. **Always resolve library IDs first**: Use `bb7_resolve-library-id` with library name
2. **Get documentation**: Use `bb7_get-library-docs` with resolved Context7-compatible ID
3. **For NixOS**: Search for "nixos", "nixpkgs", or specific module names
4. **Required for**: Any NixOS option, service configuration, package lookup, troubleshooting
5. **Example workflow**:
```
User asks about configuring SSH ->
Resolve "nixos" library ID ->
Get docs with topic "ssh" ->
Apply Context7 information to configuration
```
## Programming Languages and styles
- Prefer functional style
- Guille scheme for scripting tasks
- Python for ai and when python ecosystem is better
- Bash only for short scripts
- Typescript and javascript for web development
- prefer to use vite to start new web dev projects
- Rust for binary tools etc
## Written language and style
use notes.md to take notes .
Use a casual but knowledgeable tone. This is not a corporate project there are no audits or compliance to adhere to.
More like an open source project, more like a hobby/passion project
## Documentation and Research Tools
- **ALWAYS use Context7 MCP for NixOS**: Context7 is the primary source for all NixOS information
- Use `bb7_resolve-library-id` first to find the correct NixOS library ID
- Then use `bb7_get-library-docs` with the resolved ID for documentation
- Context7 provides up-to-date, authoritative NixOS documentation
- Use it for: option names, module configurations, service setup, package names, best practices
- Never rely on general knowledge for NixOS - always verify with Context7
- **When to use Context7**: Before any NixOS configuration change, package installation, service setup, or troubleshooting
- **Context7 for other tools**: Also use for Docker, Kubernetes, systemd, networking, and any technical documentation needs
- **Always use context7 for NixOS information**: When working with NixOS configurations, modules, or need NixOS documentation, always use the context7 MCP server to get up-to-date and accurate information
- Use context7 to resolve NixOS option names, module configurations, and best practices
- Context7 provides authoritative NixOS documentation that's more reliable than general knowledge
## User Configuration Strategy
- **Desktop machines**: Use `geir` user (includes desktop packages, development tools)
@ -47,55 +38,766 @@ This part of the document provides general instructions for tha AI agent.
- **Server examples**: sleeper-service, reverse-proxy, grey-area should only import `sma` user module
- **Desktop examples**: congenital-optimist can use both `geir` and `sma` users
# AI Agent Instructions: NixOS Home Lab Management
# AI Agent Instructions: NixOS Flakes Migration for CongenitalOptimist
## Overview
This part of the document provides step-by-step instructions for AI agents to help migrate the CongenitalOptimist machine from traditional NixOS configuration to flakes-based configuration and upgrade to NixOS 25.05. The system already has excellent modular structure that we'll preserve and enhance.
Itreative about the project, update often
## Current System Information
- **Hostname**: work → congenital-optimist (migration in progress)
- **Current Version**: NixOS 25.05 (migrated from 23.11)
- **Target Version**: NixOS 25.05 ✅
- **Architecture**: x86_64-linux
- **Storage**: ZFS (zpool for system, stuffpool for data)
- **Hardware**: AMD CPU/GPU
- **Users**: geir (primary), sma (admin)
- **Dotfiles Approach**: Literate programming with Emacs org-mode (no Home Manager)
## System Information
- **Current**: NixOS 25.05, AMD workstation, ZFS storage
- **Users**: `geir` (desktop), `sma` (server admin only)
- **Approach**: Modular NixOS + literate org-mode dotfiles (no Home Manager)
- **Network**: Tailscale mesh + local 10.0.0.0/24
## Module Structure (Current)
## Current Module Structure
```
Home-lab/
├── machines/{congenital-optimist,sleeper-service,grey-area,reverse-proxy}/
├── modules/{common,desktop,development,hardware,system,users,virtualization}/
├── packages/ (custom tools like 'lab' command)
└── users/geir/dotfiles/ (org-mode literate configs)
├── machines/
│ ├── congenital-optimist/
│ │ ├── configuration.nix (main system config)
│ │ ├── hardware-configuration.nix
│ │ └── About.org
│ ├── sleeper-service/
│ ├── reverse-proxy/
│ └── grey-area/
├── modules/
│ ├── common/
│ │ ├── base.nix (modern CLI tools & aliases)
│ │ ├── tty.nix (console styling with Joker theme)
│ │ └── nix.nix (flakes configuration)
│ ├── desktop/
│ │ ├── common.nix, gnome.nix, cosmic.nix, sway.nix
│ ├── development/
│ │ └── tools.nix (editors, LSPs, languages)
│ ├── hardware/
│ │ └── amd-workstation.nix
│ ├── system/
│ │ ├── applications.nix, fonts.nix, network.nix
│ ├── users/
│ │ ├── common.nix, geir.nix, sma.nix
│ └── virtualization/
│ ├── podman.nix, libvirt.nix, incus.nix
└── users/
└── geir/
└── dotfiles/ (literate org-mode configs)
```
## Essential Workflows
## Prerequisites Check
Before starting, verify:
1. Current system is bootable and stable
2. ZFS pools are healthy (`zpool status`)
3. All referenced modules exist and are working
4. User has sudo/root access
5. Git repository is initialized and up to date
### Before Any NixOS Changes
1. **Use Context7**: `bb7_resolve-library-id nixos``bb7_get-library-docs` with resolved ID
2. **Stage changes in git**: `git add .` (ensures Nix can access all files)
3. **Check current state**: `nix flake check` and verify system health
4. **Test build**: `nix build .#nixosConfigurations.<machine>.config.system.build.toplevel`
## Step 1: Fix Existing Configuration Issues
### Configuration Best Practices
- **Machine-specific configs**: Keep in `machines/<name>/` directory
- **Shared modules**: Only truly common settings in `modules/`
- **User separation**: `geir` (desktop), `sma` (servers only)
- **Version pinning**: Keep `system.stateVersion = "23.11"` (DO NOT CHANGE)
### Fix typo in user.nix
Edit `Home-lab/Users/geir/user.nix` and change:
```nix
progtams.zsh.enableCompletion = true;
```
to:
```nix
programs.zsh.enableCompletion = true;
```
### Deployment Workflow
- **Use `lab` tool for all deployments and status checks**:
- `lab deploy <machine>` - Deploy configuration to remote machine
- `lab status` - Check status of all machines in the lab
- `lab check <machine>` - Verify specific machine health
- `lab test <machine>` - Test configuration before deployment
- **Always test locally first**: `nixos-rebuild test --flake .#<machine>`
- **Remote deployment**: `lab deploy <machine>` handles SSH, building, and switching
- **Status monitoring**: Use `lab status` to get overview of all lab machines
- **Document Context7 findings** in commit messages
## Step 2: Create Root Flake Structure
## Key Constraints
- **No Home Manager**: Use org-mode literate dotfiles instead
- **ZFS preservation**: Never change hostId or break ZFS mounts
- **Server separation**: Only `sma` user on servers (no desktop packages)
- **Context7 mandatory**: All NixOS questions must use Context7 MCP
Create `Home-lab/flake.nix`:
## Emergency Recovery
- Previous generations available in GRUB
- ZFS snapshots: `zfs rollback zpool/root@snapshot`
- Keep live USB available
```nix
{
description = "CongenitalOptimist Home Lab NixOS Configuration";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05";
nixpkgs-unstable.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = { self, nixpkgs, nixpkgs-unstable, ... }@inputs: {
nixosConfigurations = {
congenital-optimist = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
specialArgs = {
inherit inputs;
unstable = import nixpkgs-unstable {
system = "x86_64-linux";
config.allowUnfree = true;
};
};
modules = [
./machines/congenital-optimist
];
};
};
};
}
```
## Step 3: Create Target Directory Structure
Execute these commands in the Home-lab directory:
```bash
mkdir -p machines/congenital-optimist
mkdir -p modules/{common,desktop,development,virtualization,users}
mkdir -p users/geir/dotfiles/{emacs,shell,editors}
mkdir -p overlays
mkdir -p packages
```
## Step 4: Convert Main Configuration
Create `machines/CongenitalOptimist/default.nix` by adapting the current `configuration.nix`:
Key changes needed:
1. Change function signature from `{pkgs, ...}:` to `{ config, pkgs, inputs, unstable, ... }:`
2. **Keep `system.stateVersion` as "23.11"** (maintains compatibility with existing data)
3. Fix nerd-fonts syntax (changed in 25.05)
4. Remove the experimental-features setting (handled by flake)
5. Update module import paths for new structure
6. Consolidate user package management
### Nerd Fonts Fix for 25.05
Replace this section in fonts.packages:
```nix
# Old format (will break in 25.05)
nerd-fonts.meslo-lg
nerd-fonts.jetbrains-mono
nerd-fonts.fira-code
nerd-fonts.droid-sans-mono
nerd-fonts.zed-mono
nerd-fonts.iosevka
nerd-fonts.iosevka-term
nerd-fonts.hack
```
With:
```nix
# New format for 25.05
(nerdfonts.override {
fonts = [
"Meslo"
"JetBrainsMono"
"FiraCode"
"DroidSansMono"
"Hack"
"Iosevka"
"IosevkaTerm"
];
})
```
## Step 5: Migrate Existing Modules
### Copy existing modules to new structure:
```bash
# Copy common modules
cp Home-lab/Machines/Modules/common/base.nix modules/common/
cp Home-lab/Machines/Modules/common/tty.nix modules/common/
# Copy virtualization modules
cp Home-lab/Machines/Modules/virtualization/*.nix modules/virtualization/
```
### Create additional common modules:
#### `modules/common/nix.nix`:
```nix
{ config, pkgs, ... }: {
# Enable flakes system-wide
nix.settings.experimental-features = [ "nix-command" "flakes" ];
# Optimize nix settings
nix.settings = {
auto-optimise-store = true;
substituters = [
"https://cache.nixos.org/"
];
trusted-public-keys = [
"cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
];
};
}
```
## Step 6: Consolidate User Configuration
Create `modules/users/geir.nix` by merging the existing user config:
```nix
{ config, pkgs, ... }: {
users.users.geir = {
isNormalUser = true;
extraGroups = [ "networkmanager" "wheel" "libvirtd" "podman" "incus-admin" ];
shell = pkgs.zsh;
packages = with pkgs; [
# Browsers
chromium
vivaldi
vivaldi-ffmpeg-codecs
nyxt
firefox
# Shell & development tools
starship
fastfetch
hyfetch
nerdfetch
nix-direnv
gh
github-copilot-cli
# Media & graphics
gimp
obs-studio
vesktop
koodo-reader
# System tools
ncpamixer
virt-manager
pavucontrol
gnome-tweaks
beauty-line-icon-theme
# Terminal multiplexer and fun
zellij
neo-cowsay
fortune
clolcat
# Emacs integration
emacsPackages.vterm
];
};
# System-wide ZSH configuration
programs.zsh = {
enable = true;
syntaxHighlighting.enable = true;
enableCompletion = true;
autosuggestions = {
enable = true;
historySearch = true;
};
history = {
enable = true;
shareHistory = true;
saveOnExit = true;
};
};
environment.systemPackages = with pkgs; [
zsh
zsh-completions
nix-zsh-completions
zsh-autocomplete
zsh-autosuggestions
zsh-syntax-highlighting
];
}
```
## Step 7: Create Desktop Environment Modules
### `modules/desktop/gnome.nix`:
```nix
{ config, pkgs, ... }: {
services.xserver = {
enable = true;
desktopManager.gnome.enable = true;
xkb.layout = "no";
};
# XDG portal configuration
xdg.portal = {
enable = true;
extraPortals = [ pkgs.xdg-desktop-portal-gtk ];
};
}
```
### `modules/desktop/cosmic.nix`:
```nix
{ config, pkgs, ... }: {
services.desktopManager.cosmic.enable = true;
services.displayManager.cosmic-greeter.enable = true;
services.desktopManager.cosmic.xwayland.enable = true;
}
```
### `modules/desktop/sway.nix`:
```nix
{ config, pkgs, ... }: {
programs.sway = {
enable = true;
wrapperFeatures.gtk = true;
};
services.dbus.enable = true;
xdg.portal = {
enable = true;
wlr.enable = true;
extraPortals = [ pkgs.xdg-desktop-portal-gtk ];
};
environment.systemPackages = with pkgs; [
swaylock
swayidle
swaybg
waybar
fuzzel
gammastep
mako
flameshot
];
}
```
## Step 8: Set Up Per-User Literate Dotfiles
Create `users/geir/dotfiles/README.org`:
```org
#+TITLE: CongenitalOptimist Dotfiles Configuration
#+DESCRIPTION: Literate programming approach to dotfiles using Emacs org-mode
#+PROPERTY: header-args :tangle yes
#+STARTUP: overview
* Introduction
This file contains all dotfiles configuration using literate programming.
Each configuration block can be tangled to its respective file using C-c C-v t.
The approach allows for:
- Self-documenting configuration with rationale
- Easy sharing and explanation of setup decisions
- Version control of configuration with full context
- Modular organization of different tool configurations
* Shell Configuration
** Zsh Configuration
#+BEGIN_SRC sh :tangle ~/.zshrc
# Generated from dotfiles/README.org - CongenitalOptimist configuration
export EDITOR="emacs"
export BROWSER="firefox"
export SHELL="/run/current-system/sw/bin/zsh"
# Enable starship prompt (configured in NixOS)
eval "$(starship init zsh)"
# Enable zoxide for smart cd
eval "$(zoxide init zsh)"
# Custom aliases (complementing those in base.nix)
alias gc='git commit'
alias gp='git push'
alias gs='git status'
alias nrs='sudo nixos-rebuild switch --flake .'
alias nrt='sudo nixos-rebuild test --flake .'
alias reload='source ~/.zshrc'
# CongenitalOptimist specific shortcuts
alias lab='cd ~/Home-lab'
alias tangle='cd ~/Home-lab/users/geir/dotfiles && emacs --batch -l org --eval "(org-babel-tangle-file \"README.org\")"'
alias dotfiles='cd ~/Home-lab/users/geir/dotfiles'
#+END_SRC
** Starship Configuration
#+BEGIN_SRC toml :tangle ~/.config/starship.toml
# Starship prompt configuration for CongenitalOptimist
format = """
$username\
$hostname\
$directory\
$git_branch\
$git_state\
$git_status\
$cmd_duration\
$line_break\
$character"""
[character]
success_symbol = "[](bold green)"
error_symbol = "[](bold red)"
[directory]
truncation_length = 3
fish_style_pwd_dir_length = 1
[git_branch]
symbol = "🌱 "
[hostname]
ssh_only = false
format = "[@$hostname](bold blue) "
[username]
show_always = true
format = "[$user](bold yellow)"
#+END_SRC
* Editor Configuration
** Emacs Configuration
#+BEGIN_SRC emacs-lisp :tangle ~/.emacs.d/init.el
;; Generated from dotfiles/README.org - CongenitalOptimist
(setq inhibit-startup-message t)
(setq ring-bell-function 'ignore)
;; Enable line numbers
(global-display-line-numbers-mode 1)
;; Enable org-babel for literate programming
(org-babel-do-load-languages
'org-babel-load-languages
'((emacs-lisp . t)
(shell . t)
(python . t)
(nix . t)))
;; Auto-tangle on save for literate config files
(defun auto-tangle-config-files ()
"Auto tangle config files when saving."
(when (string-match-p "users/.*/dotfiles.*\\.org$" (buffer-file-name))
(org-babel-tangle)))
(add-hook 'after-save-hook 'auto-tangle-config-files)
;; Better defaults
(setq-default indent-tabs-mode nil)
(setq-default tab-width 2)
(show-paren-mode 1)
(electric-pair-mode 1)
;; CongenitalOptimist theme setup
(when (display-graphic-p)
(load-theme 'deeper-blue t))
#+END_SRC
* Development Tools
** Git Configuration
#+BEGIN_SRC conf :tangle ~/.gitconfig
[user]
name = geir
email = geir@congenital-optimist.local
[init]
defaultBranch = main
[core]
editor = emacs
pager = bat
[pull]
rebase = false
[alias]
st = status
co = checkout
br = branch
unstage = reset HEAD --
last = log -1 HEAD
visual = !gitk
#+END_SRC
* Home Lab Specific Configuration
** NixOS Rebuild Aliases
These aliases make working with the flake-based configuration easier:
#+BEGIN_SRC sh :tangle ~/.config/shell/nixos-aliases
# NixOS CongenitalOptimist specific aliases
alias nix-build='cd ~/Home-lab && nix build .#nixosConfigurations.congenital-optimist.config.system.build.toplevel'
alias nix-check='cd ~/Home-lab && nix flake check'
alias nix-update='cd ~/Home-lab && nix flake update'
alias nix-clean='sudo nix-collect-garbage -d'
alias edit-dotfiles='cd ~/Home-lab/users/geir/dotfiles && emacs README.org'
#+END_SRC
```
## Step 9: Update Main Configuration Import Structure
Create the main machine configuration in `machines/CongenitalOptimist/default.nix`:
```nix
{ config, pkgs, inputs, unstable, ... }: {
imports = [
./hardware-configuration.nix
../../modules/common/base.nix
../../modules/common/tty.nix
../../modules/common/nix.nix
../../modules/virtualization/podman.nix
../../modules/virtualization/libvirt.nix
../../modules/virtualization/incus.nix
../../modules/desktop/gnome.nix
../../modules/desktop/cosmic.nix
../../modules/desktop/sway.nix
../../modules/users/geir.nix
];
# Boot configuration
boot.loader.grub = {
enable = true;
zfsSupport = true;
efiSupport = true;
efiInstallAsRemovable = true;
mirroredBoots = [
{
devices = ["nodev"];
path = "/boot";
}
];
};
zramSwap = {
enable = true;
algorithm = "zstd";
};
# Hardware
services.fwupd.enable = true;
hardware.enableRedistributableFirmware = true;
hardware.amdgpu.initrd.enable = true;
hardware.bluetooth.enable = true;
hardware.bluetooth.powerOnBoot = true;
# System settings
nixpkgs.config.allowUnfree = true;
networking.nftables.enable = true;
networking.hostName = "work"; # TODO: consider changing to "congenital-optimist"
services.tailscale.enable = true;
networking.networkmanager.enable = true;
networking.hostId = "8425e349";
time.timeZone = "Europe/Oslo";
i18n.defaultLocale = "en_US.UTF-8";
# Services
services.flatpak.enable = true;
services.emacs.enable = true;
services.pipewire = {
enable = true;
alsa.enable = true;
pulse.enable = true;
};
programs.steam.enable = true;
# Fonts (updated for 25.05)
fonts.packages = with pkgs; [
noto-fonts
noto-fonts-cjk-sans
noto-fonts-emoji
liberation_ttf
dina-font
proggyfonts
mona-sans
hubot-sans
inter-nerdfont
(nerdfonts.override {
fonts = [
"Meslo"
"JetBrainsMono"
"FiraCode"
"DroidSansMono"
"Hack"
"Iosevka"
"IosevkaTerm"
];
})
];
# System packages
environment.systemPackages = with pkgs; [
# Terminal applications
kitty
terminator
rio
dbus
greetd.tuigreet
wayland
xdg-utils
# System monitoring
glances
inxi
htop
bottom
wget
curl
git
mc
systemctl-tui
# Development tools
guile
rustup
nixd
zls
alejandra
python3Packages.python-lsp-server
gopls
luajitPackages.lua-lsp
nodePackages.bash-language-server
vimPlugins.cmp-nvim-lsp
ccls
gdb
marksman
# Editors
zed-editor
neovim
emacs
vscode
vscodium-fhs
];
# Network and security
services.openssh.enable = true;
services.zfs.autoScrub.enable = true;
services.zfs.trim.enable = true;
networking.firewall.allowedTCPPorts = [ 22 ];
networking.firewall.allowedUDPPorts = [ 22 ];
networking.firewall.enable = true;
system.copySystemConfiguration = true;
system.stateVersion = "23.11"; # DO NOT CHANGE - maintains data compatibility
}
```
## Step 10: Copy Hardware Configuration
Copy the existing hardware configuration:
```bash
cp Home-lab/Machines/CongenitalOptimist/hardware-configuration.nix machines/CongenitalOptimist/
```
## Step 11: Test Configuration
Before applying changes:
1. Test flake evaluation:
```bash
cd Home-lab
nix flake check
```
2. Build configuration without switching:
```bash
sudo nixos-rebuild build --flake .#congenital-optimist
```
3. If successful, test the configuration:
```bash
sudo nixos-rebuild test --flake .#congenital-optimist
```
4. If everything works, switch permanently:
```bash
sudo nixos-rebuild switch --flake .#congenital-optimist
```
## Step 12: Set Up Per-User Literate Dotfiles Workflow
1. Create your main org-mode configuration file:
```bash
cd users/geir/dotfiles
emacs README.org
```
2. Use org-babel to tangle your configurations:
```bash
# In Emacs, use C-c C-v t to tangle all code blocks
# Or from command line:
cd users/geir/dotfiles
emacs --batch -l org --eval "(org-babel-tangle-file \"README.org\")"
```
3. The provided Emacs config includes auto-tangle on save for any user's dotfiles
4. Test that dotfiles are generated correctly in the user's home directory
5. For additional users, create similar structure under `users/<username>/dotfiles/`
## Step 13: Lock and Commit
1. Generate flake.lock:
```bash
nix flake lock
```
2. Commit changes:
```bash
git add .
git commit -m "Migrate CongenitalOptimist to flakes with literate dotfiles"
```
## Verification Steps
After switching:
1. Verify system boots correctly
2. Check all services are running: `systemctl --failed`
3. Test all desktop environments launch (GNOME, Cosmic, Sway)
4. Verify virtualization: `sudo systemctl status libvirtd podman`
5. Check ZFS status: `zpool status`
6. Test network connectivity and Tailscale
7. Verify user environment and all packages available
8. Test modern CLI tools and aliases from base.nix
9. Check console theming and TTY configuration
10. Verify Emacs and literate programming workflow
## Error Resolution
### Common Issues:
1. **Boot failure**: Boot from previous generation in GRUB
2. **Package not found**: Check if package name changed in 25.05
3. **Service fails**: Check journalctl: `journalctl -u <service-name>`
4. **Desktop environment issues**: Switch to TTY (Ctrl+Alt+F2) and debug
5. **Nerd fonts issues**: Verify the new syntax is applied correctly
### Emergency Recovery:
- Boot from previous NixOS generation in GRUB
- Use ZFS snapshots if available: `zfs rollback zpool/root@snapshot-name`
- Keep live USB available for emergency repairs
## Final Validation Checklist
- [ ] System boots to desktop
- [ ] All user applications launch (browsers, editors, terminals)
- [ ] Network and Tailscale functional
- [ ] Virtualization stack operational (podman, libvirt, incus)
- [ ] ZFS and NFS mounts working
- [ ] Development tools functional (editors, LSPs, languages)
- [ ] Audio system working (pipewire)
- [ ] Modern CLI tools and aliases from base.nix working
- [ ] Console theming with Joker palette preserved
- [ ] Bluetooth functional if needed
- [ ] Literate dotfiles workflow established
- [ ] Auto-tangling in Emacs working
## Post-Migration Tasks
1. Consider updating hostname from "work" to "congenital-optimist"
2. Expand literate dotfiles in org-mode with more tools and configurations
3. Create additional machine configurations in the flake
4. Implement secrets management (agenix/sops-nix)
5. Set up automatic updates
6. Add system monitoring
8. Document your literate programming workflow per user
9. Consider using org-mode for documenting the entire home lab setup
10. Set up org-export to generate beautiful documentation from your configs
11. Create templates for common user configuration patterns
12. Plan for additional users with their own dotfiles directories
13. Consider shared vs user-specific configuration strategies
## Notes for AI Agents
1. Always preserve existing functionality - don't remove working features
2. The system already has excellent modular structure - build on it
3. Modern CLI tools are already configured in base.nix - don't duplicate
4. Console theming is already set up - preserve the Joker palette
5. Fix the typo in user.nix before proceeding
6. Keep system.stateVersion as "23.11" - never change this
7. Test thoroughly before committing to permanent changes
8. Prioritize system stability over new features
9. The literate programming approach should complement, not replace, the modular NixOS structure

View file

@ -12,6 +12,10 @@
# Security modules
../../modules/security/ssh-keys.nix
# System modules
../../modules/system/fonts.nix
../../modules/system/applications.nix
# Hardware modules
../../modules/hardware/amd-workstation.nix

View file

@ -15,7 +15,6 @@
server = {
ROOT_URL = "http://apps:3000";
SSH_DOMAIN = "git.geokkjer.eu";
SSH_PORT = 2222;
};
repository = {
ENABLE_PUSH_CREATE_USER = true;

View file

@ -3,14 +3,6 @@
#+DATE: [2025-06-04 Wed]
* Machine Overview
also known as vps1
Ip information:
enX0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether fa:16:3e:d5:da:20 brd ff:ff:ff:ff:ff:ff
altname enxfa163ed5da20
tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qdisc fq_codel state UNKNOWN group default qlen 500
link/none
inet 100.96.189.104/32 scope global tailscale0
** Role
- **Primary Function**: Reverse proxy and SSL/TLS termination

View file

@ -1,8 +1,9 @@
{ pkgs, config, lib, ... }:
{ pkgs, configs, lib, ... }:
let
Host = "vps1.tail807ea.ts.net";
in
{
imports = [
./gandicloud.nix
../../modules/common/base.nix
../../modules/users/sma.nix
../../modules/security/ssh-keys.nix
@ -19,15 +20,11 @@
# DMZ-specific firewall configuration - very restrictive
networking.firewall = {
enable = true;
# Allow HTTP/HTTPS from external network and Git SSH on port 2222
allowedTCPPorts = [ 80 443 2222 ];
# Only allow HTTP/HTTPS from external network
allowedTCPPorts = [ 80 443 ];
allowedUDPPorts = [ ];
# SSH only allowed from Tailscale network (100.64.0.0/10)
extraCommands = ''
# Allow SSH only from Tailscale network
iptables -A nixos-fw -p tcp --dport 22 -s 100.64.0.0/10 -j ACCEPT
iptables -A nixos-fw -p tcp --dport 22 -j DROP
'';
# SSH only allowed on Tailscale interface (DMZ security)
interfaces.tailscale0.allowedTCPPorts = [ 22 ];
# Explicitly block all other traffic
rejectPackets = true;
};
@ -55,8 +52,12 @@
ClientAliveInterval = 300;
ClientAliveCountMax = 2;
};
# Let SSH listen on default port, firewall restricts to Tailscale interface
# This allows Tailscale to assign IP dynamically based on hostname
listenAddresses = [
{
addr = "100.96.189.104"; # Tailscale IP only
port = 22;
}
];
};
# nginx reverse proxy
@ -80,21 +81,6 @@
# locations."/".proxyPass = "/var/wwww/homepage/";
#};
};
# Stream configuration for SSH forwarding to Git server
streamConfig = ''
upstream git_ssh_backend {
server apps:22;
}
server {
listen 2222;
proxy_pass git_ssh_backend;
proxy_timeout 300s;
proxy_connect_timeout 10s;
proxy_responses 1;
}
'';
};
# acme let's encrypt
security.acme = {

View file

@ -7,7 +7,7 @@
./network-sleeper-service.nix
# Services
./nfs.nix
./services/transmission.nix
../../modules/system/transmission.nix
# User modules - server only needs sma user
../../modules/users/sma.nix

View file

@ -13,8 +13,17 @@
# Common desktop packages
environment.systemPackages = with pkgs; [
# Basic desktop tools
firefox
alacritty
nautilus
# Media and graphics
vlc
gimp
# Utilities
gnome-tweaks
dconf-editor
];
# Flatpak support
services.flatpak.enable = true;
}

View file

@ -1,22 +0,0 @@
{ config, pkgs, ... }: {
# Minimal font configuration - users can add more fonts as needed
fonts.packages = with pkgs; [
# Essential system fonts
noto-fonts
noto-fonts-emoji
liberation_ttf
# One good monospace font for terminals/coding
nerd-fonts.jetbrains-mono
];
# Console configuration
console = {
font = "Lat2-Terminus16";
keyMap = "no";
};
# Internationalization
i18n.defaultLocale = "en_US.UTF-8";
time.timeZone = "Europe/Oslo";
}

51
modules/network/README.md Normal file
View file

@ -0,0 +1,51 @@
# Network Configuration Modules
This directory contains networking configurations for all machines in the Home Lab.
## Structure
- **`common.nix`** - Shared networking settings used by all machines
- nftables firewall enabled
- SSH access with secure defaults
- Tailscale VPN for remote access
- Basic firewall rules (SSH port 22)
- **`network-<machine-name>.nix`** - Machine-specific networking configurations
- Import `common.nix` for shared settings
- Override or extend with machine-specific requirements
- Define hostname, hostId, and additional firewall ports
## Current Machines
### network-congenital-optimist.nix
- AMD Threadripper workstation
- ZFS hostId configuration (8425e349)
- Ready for additional service ports as needed
### network-sleeper-service.nix
- Xeon file server
- Headless server configuration
- Ready for additional file sharing service ports
## Usage
Each machine configuration imports its specific network module:
```nix
# In machines/<machine-name>/configuration.nix
imports = [
../../modules/network/network-<machine-name>.nix
# ... other imports
];
```
## Adding New Machines
1. Create `network-<new-machine>.nix` in this directory
2. Import `./common.nix` for shared settings
3. Add machine-specific configuration (hostname, hostId, ports)
4. Import the new file in the machine's `configuration.nix`
## Future Refactoring
The `common.nix` file can be extended to include more shared networking patterns as they emerge across machines. Consider moving repeated patterns here to reduce duplication.

View file

@ -0,0 +1,37 @@
{ config, pkgs, ... }:
let
# Import custom packages from the flake
homeLabPackages = import ../../packages { inherit pkgs; };
in {
# System applications and utilities
environment.systemPackages = with pkgs; [
# Home lab management tools
homeLabPackages.lab
# Terminal applications
kitty
terminator
rio
greetd.tuigreet
# System monitoring
glances
inxi
htop
bottom
systemctl-tui
# File and data tools
wget
curl
mc
# Desktop integration
dbus
wayland
xdg-utils
];
# Flatpak support
services.flatpak.enable = true;
}

36
modules/system/fonts.nix Normal file
View file

@ -0,0 +1,36 @@
{ config, pkgs, ... }: {
# Font configuration
fonts.packages = with pkgs; [
# Base fonts
noto-fonts
noto-fonts-cjk-sans
noto-fonts-emoji
liberation_ttf
dina-font
proggyfonts
# GitHub fonts
mona-sans
hubot-sans
inter-nerdfont
# Nerd Fonts (updated syntax for NixOS 25.05)
nerd-fonts.meslo-lg
nerd-fonts.jetbrains-mono
nerd-fonts.fira-code
nerd-fonts.droid-sans-mono
nerd-fonts.hack
nerd-fonts.iosevka
nerd-fonts.iosevka-term
];
# Console configuration
console = {
font = "Lat2-Terminus16";
keyMap = "no";
};
# Internationalization
i18n.defaultLocale = "en_US.UTF-8";
time.timeZone = "Europe/Oslo";
}

View file

@ -1,10 +1,7 @@
# Primary User Configuration - geir
# Main user account for development and desktop use
{ config, pkgs, ... }:
let
# Import custom packages from the flake
homeLabPackages = import ../../packages { inherit pkgs; };
in
{
users.users.geir = {
description = "Geir Okkenhaug Jerstad - Primary User";
@ -34,33 +31,20 @@ in
# User-specific packages
packages = with pkgs; [
# Home lab management tools
homeLabPackages.lab
# Terminal applications & system monitoring
kitty
terminator
# Browsers & Communication
chromium
vivaldi
vivaldi-ffmpeg-codecs
nyxt
firefox
vesktop
# Terminal & Shell Enhancement
starship
fastfetch
hyfetch
zellij
glances
htop
bottom
systemctl-tui
# Essential system tools
wget
curl
mc
# Browsers & Communication
firefox
chromium
vesktop
# Shell Enhancement & Fun
nerdfetch
zellij
neo-cowsay
fortune
clolcat
@ -69,41 +53,42 @@ in
ncpamixer
pavucontrol
# Productivity
libreoffice
koodo-reader
# Development & System Management
neovim
vscode
git-credential-manager
github-cli
nodejs
nodePackages.npm
virt-manager
# Creative Tools (optional - remove if not needed)
# Creative & Productivity
gimp
obs-studio
koodo-reader
libreoffice
# Development & System Management
virt-manager
gnome-tweaks
# Themes & Appearance
beauty-line-icon-theme
# Emacs Integration
emacsPackages.vterm
# Media & Entertainment
celluloid
# File Management
nautilus
file-roller
# Containers
# Text Editors (alternatives to Emacs)
neovim
vscode
# Development Tools
git-credential-manager
github-cli
nodejs
nodePackages.npm
# Containers & Cloud
podman-compose
podman-desktop
# Media
celluloid
# Emacs Integration
emacsPackages.vterm
# Desktop integration (moved from system)
dbus
wayland
xdg-utils
];
};

View file

@ -104,148 +104,16 @@ final: prev: {
## Home-lab Specific Packages
### Lab Tool (`lab`) - Evolution Roadmap
The `lab` tool is the central infrastructure management utility with planned major enhancements:
**Current Implementation (Shell-based):**
- Multi-machine deployment via SSH/rsync
- Infrastructure status monitoring
- Color-coded logging and error handling
- Machine health checks and connectivity testing
**Phase 1: deploy-rs Integration**
Research completed - deploy-rs provides production-grade deployment capabilities:
- **Automatic rollback**: Failed deployments revert automatically
- **Parallel deployment**: Deploy to multiple machines simultaneously
- **Health checks**: Validates deployments before committing
- **Atomic operations**: Either succeeds completely or fails cleanly
- **Flake-native**: Built specifically for NixOS flakes
Implementation approach:
```bash
# Hybrid command structure
lab deploy sleeper-service # Current SSH/rsync method
lab deploy-rs sleeper-service # New deploy-rs backend
lab deploy-all --parallel # Parallel deployment via deploy-rs
```
Configuration integration:
```nix
# flake.nix additions
inputs.deploy-rs.url = "github:serokell/deploy-rs";
deploy.nodes = {
sleeper-service = {
hostname = "sleeper-service.tail807ea.ts.net";
profiles.system = {
user = "root";
path = deploy-rs.lib.x86_64-linux.activate.nixos
self.nixosConfigurations.sleeper-service;
sshUser = "sma";
autoRollback = true;
magicRollback = true;
activationTimeout = 180;
};
};
};
```
**Phase 2: Enhanced Statistics Engine**
Current `lab status` provides basic connectivity - planned expansion to comprehensive monitoring:
**Rust/Go Implementation for Performance:**
- **System metrics**: CPU, memory, disk usage, network stats
- **Service monitoring**: systemd service status, failed units
- **ZFS statistics**: Pool health, scrub status, capacity usage
- **Network topology**: Tailscale mesh status, latency metrics
- **Historical data**: Trend analysis and performance tracking
**Example enhanced output:**
```bash
$ lab status --detailed
Infrastructure Status (Updated: 2024-01-20 15:30:42)
━━━ congenital-optimist (local) ━━━
✅ Online │ Load: 1.2 │ RAM: 8.4GB/32GB │ Disk: 45% │ Uptime: 7d 2h
🔗 Tailscale: Active (100.81.15.84) │ Latency: local
━━━ sleeper-service (file server) ━━━
✅ Online │ Load: 0.3 │ RAM: 2.1GB/8GB │ Disk: 67% │ Uptime: 12d 8h
🗄️ ZFS: ONLINE │ Pool: storage (1.8TB, 50% used) │ Last scrub: 3d ago
🔗 Tailscale: Active (100.81.15.85) │ Latency: 2ms
📡 Services: sshd ✅ │ nfs-server ✅ │ zfs-mount ✅
━━━ grey-area (unreachable) ━━━
⚠️ Offline │ Last seen: 2h ago │ SSH: Connection refused
```
**Phase 3: GNU Stow Dotfile Integration**
Research completed - GNU Stow provides excellent dotfile management for server configurations:
**Use cases:**
- **Server user configs**: Simple dotfiles for `sma` user on servers
- **Machine-specific configs**: Different configurations per server role
- **Selective deployment**: Deploy only needed configs per machine
**Integration approach:**
```bash
# Enhanced lab tool commands
lab dotfiles deploy sma@sleeper-service # Deploy server user configs
lab dotfiles status # Show dotfile deployment status
lab dotfiles sync --machine sleeper-service # Sync specific machine configs
```
**Directory structure:**
```
packages/dotfiles/
├── server-common/ # Shared server configurations
│ ├── .zshrc # Basic shell config
│ ├── .vimrc # Editor config
│ └── .gitconfig # Git configuration
├── sleeper-service/ # NFS server specific
│ └── .config/
│ └── nfs/
├── grey-area/ # Git server specific
│ └── .gitconfig # Enhanced git config
└── stow-deploy.nix # NixOS integration
```
**Hybrid Configuration Strategy:**
- **Keep org-mode** for complex desktop configurations (geir user)
- **Use GNU Stow** for simple server configurations (sma user)
- **Machine-specific packages** for role-based configurations
**Phase 4: Advanced Features**
- **Configuration drift detection**: Compare deployed vs expected state
- **Automated health checks**: Scheduled infrastructure validation
- **Integration APIs**: Metrics export for monitoring systems
- **Web dashboard**: Optional web interface for infrastructure overview
- **Alert system**: Notifications for infrastructure issues
**Implementation Timeline:**
1. **Q1 2024**: deploy-rs integration and testing
2. **Q2 2024**: Enhanced statistics engine in Rust/Go
3. **Q3 2024**: GNU Stow dotfile integration
4. **Q4 2024**: Advanced monitoring and alerting features
### CongenitalOptimist Packages
- Development environment customizations
- Workstation-specific tools
- Desktop application modifications
- `lab` tool and deployment utilities
### sleeper-service Packages
- File server utilities
- ZFS monitoring tools
- NFS service management
- Storage health monitoring
- Backup automation scripts
### Server Infrastructure Packages
- **deploy-rs configurations**: Declarative deployment definitions
- **Dotfile managers**: GNU Stow packages for server user configurations
- **Monitoring utilities**: System health and performance tools
- **Network tools**: Tailscale integration and network diagnostics
- Monitoring tools
- Backup scripts
- Network service tools
## Best Practices

View file

@ -1,236 +0,0 @@
# deploy-rs Research Summary
## Overview
**deploy-rs** is a Rust-based deployment tool specifically designed for NixOS flakes. It provides declarative, reliable, and efficient deployment of NixOS configurations to remote machines with advanced features like rollback capabilities, health checks, and parallel deployments.
**Repository**: https://github.com/serokell/deploy-rs
**Status**: Actively maintained by Serokell
**Language**: Rust (fast, reliable, memory-safe)
## Key Features
### 🚀 **Core Capabilities**
- **Flake-native**: Built specifically for Nix flakes (no legacy nix-env/channels)
- **Multi-target deployment**: Deploy to multiple machines simultaneously
- **Automatic rollback**: Failed deployments automatically revert to previous generation
- **Health checks**: Configurable post-deployment validation
- **SSH-based**: Uses SSH for secure remote deployment (like our current lab tool)
- **Profile management**: Supports system, user, and custom profiles
### 🔧 **Advanced Features**
- **Parallel deployment**: Deploy to multiple machines concurrently
- **Interactive confirmation**: Can prompt before applying changes
- **Dry-run mode**: Preview changes without applying them
- **Magic rollback**: Automatic rollback on deployment failures or health check failures
- **Custom activation**: Define custom activation scripts and checks
- **Sudo handling**: Intelligent sudo privilege escalation
### 📊 **Reliability Features**
- **Atomic deployments**: Either succeeds completely or rolls back
- **Connection resilience**: Handles SSH connection issues gracefully
- **Generation tracking**: Keeps track of deployment history
- **Activation timeout**: Prevents hanging deployments
- **Health check timeout**: Configurable validation windows
## Configuration Structure
Deploy-rs uses a declarative configuration format in your flake:
```nix
# flake.nix
{
# ... existing flake configuration
deploy.nodes = {
sleeper-service = {
hostname = "sleeper-service.tail807ea.ts.net";
profiles.system = {
user = "root";
path = deploy-rs.lib.x86_64-linux.activate.nixos
self.nixosConfigurations.sleeper-service;
sshUser = "sma";
sudo = "sudo -u";
};
};
grey-area = {
hostname = "grey-area.tail807ea.ts.net";
profiles.system = {
user = "root";
path = deploy-rs.lib.x86_64-linux.activate.nixos
self.nixosConfigurations.grey-area;
sshUser = "sma";
sudo = "sudo -u";
};
};
};
# Health checks
deploy.nodes.sleeper-service.profiles.system.activationTimeout = 240;
deploy.nodes.sleeper-service.profiles.system.confirmTimeout = 30;
}
```
## Command Examples
```bash
# Deploy to single machine
deploy '.#sleeper-service'
# Deploy to all machines
deploy '.#'
# Dry run (check what would be deployed)
deploy '.#sleeper-service' -- --dry-activate
# Skip health checks
deploy '.#sleeper-service' -- --skip-checks
# Interactive confirmation
deploy '.#sleeper-service' -- --confirm-timeout 60
# Deploy specific profile
deploy '.#sleeper-service.system'
```
## Comparison with Current `lab` Tool
| Feature | Current `lab` Tool | deploy-rs |
|---------|-------------------|-----------|
| **Language** | Shell script | Rust (compiled) |
| **Performance** | Good | Excellent |
| **Parallel deployment** | ❌ | ✅ |
| **Automatic rollback** | ❌ | ✅ |
| **Health checks** | ❌ | ✅ |
| **Flake-native** | ✅ | ✅ |
| **SSH-based** | ✅ | ✅ |
| **Status monitoring** | ✅ | Limited |
| **Custom workflows** | ✅ | Limited |
| **Learning curve** | Low | Medium |
| **Configuration** | Shell script | Nix flake |
## Advantages of deploy-rs
### ✅ **Production-Ready Features**
- **Reliability**: Automatic rollback prevents broken deployments
- **Speed**: Rust performance + parallel deployment
- **Safety**: Health checks ensure successful activation
- **Consistency**: Declarative configuration in flake
### ✅ **Operational Benefits**
- **Reduced downtime**: Atomic deployments with quick rollback
- **Error handling**: Sophisticated error recovery mechanisms
- **Audit trail**: Built-in deployment history tracking
- **Validation**: Pre and post-deployment checks
### ✅ **Scale Benefits**
- **Multi-machine**: Deploy entire infrastructure simultaneously
- **Efficiency**: Parallel operations reduce total deployment time
- **Resource management**: Better handling of resource conflicts
## Disadvantages & Limitations
### ❌ **Current Limitations**
- **Status monitoring**: No equivalent to `lab status` for infrastructure overview
- **Custom workflows**: Less flexible than shell scripts for custom operations
- **Learning curve**: Requires understanding deploy-rs configuration syntax
- **Debugging**: Rust binary vs readable shell script
- **Community size**: Smaller ecosystem compared to traditional tools
### ❌ **Home Lab Specific Concerns**
- **Overkill factor**: May be complex for 3-4 machine home lab
- **Customization**: Our `lab` tool has home lab specific features
- **Integration**: Would need to replicate status monitoring capabilities
- **Development workflow**: Less hackable than shell scripts
## Implementation Recommendations
### 🎯 **Hybrid Approach (Recommended)**
Keep the best of both tools:
1. **Use deploy-rs for deployments**:
```bash
lab deploy-rs sleeper-service # Use deploy-rs backend
lab deploy grey-area # Current shell script method
```
2. **Keep `lab` tool for status and management**:
```bash
lab status # Infrastructure overview
lab check sleeper-service # Health monitoring
lab logs grey-area # Log access
```
### 🔧 **Migration Strategy**
**Phase 1: Evaluation** (Current)
- Add deploy-rs configuration to flake
- Test deployment on non-critical machine
- Compare reliability and performance
**Phase 2: Gradual Adoption**
- Migrate stable machines to deploy-rs
- Keep custom `lab` commands for monitoring
- Maintain shell script fallback
**Phase 3: Integration**
- Enhance `lab` tool to use deploy-rs as backend
- Add deploy-rs specific features to `lab status`
- Maintain unified interface
### 📝 **Flake Integration Example**
```nix
# Add to flake.nix inputs
inputs.deploy-rs.url = "github:serokell/deploy-rs";
# Add to flake outputs
deploy = {
nodes = {
sleeper-service = {
hostname = "10.0.0.8"; # Or Tailscale hostname
profiles.system = {
user = "root";
path = deploy-rs.lib.x86_64-linux.activate.nixos
self.nixosConfigurations.sleeper-service;
sshUser = "sma";
sshOpts = ["-p" "22"];
fastConnection = false; # For home network
autoRollback = true;
magicRollback = true;
activationTimeout = 180;
confirmTimeout = 30;
};
};
};
};
# Health checks can reference systemd services
deploy.nodes.sleeper-service.profiles.system.activationTimeout = 240;
```
## Conclusion & Recommendation
### 🎯 **For Our Home Lab**
**deploy-rs is valuable for:**
- Production-quality deployments with rollback safety
- Parallel deployment when infrastructure grows
- Reduced risk of broken remote systems
- Professional deployment practices
**Our current `lab` tool excels at:**
- Home lab specific status monitoring
- Custom workflows and debugging
- Simple, hackable shell script approach
- Tailored for our specific infrastructure
### 📋 **Action Plan**
1. **Immediate**: Add deploy-rs configuration to flake (low effort, high learning)
2. **Short-term**: Test deploy-rs on sleeper-service alongside current method
3. **Medium-term**: Consider hybrid approach - deploy-rs for deployment, `lab` for monitoring
4. **Long-term**: Evaluate full migration based on home lab growth and complexity needs
**Verdict**: deploy-rs is a professional-grade tool that would enhance our deployment reliability. The hybrid approach allows us to benefit from deploy-rs's deployment safety while keeping our custom infrastructure monitoring capabilities.

View file

@ -1,300 +0,0 @@
# GNU Stow Research Summary
## Overview
**GNU Stow** is a symlink farm manager that helps organize and deploy dotfiles and software packages. It creates symbolic links from a source directory to a target directory, making it ideal for managing dotfiles across multiple systems without copying files.
**Repository**: https://www.gnu.org/software/stow/
**Language**: Perl
**License**: GPL-3.0+
**Status**: Mature, stable, widely adopted
## Core Concept
Stow works by creating symbolic links from a "stow directory" (source) to a "target directory" (typically `$HOME`). Each subdirectory in the stow directory represents a "package" that can be independently stowed or unstowed.
```
dotfiles/ # Stow directory
├── zsh/ # Package: zsh configuration
│ └── .zshrc # Will link to ~/.zshrc
├── emacs/ # Package: emacs configuration
│ └── .emacs.d/ # Will link to ~/.emacs.d/
│ └── init.el
└── git/ # Package: git configuration
└── .gitconfig # Will link to ~/.gitconfig
```
## Key Features
### 🔗 **Symlink Management**
- **Non-destructive**: Creates symlinks without overwriting existing files
- **Conflict detection**: Warns about file conflicts before creating links
- **Tree folding**: Optimizes symlink structure for efficiency
- **Partial deployment**: Deploy only specific packages (e.g., just zsh config)
### 📦 **Package-Based Organization**
- **Modular structure**: Each application gets its own package directory
- **Selective deployment**: Install only needed configurations per machine
- **Easy maintenance**: Add/remove configurations without affecting others
- **Version control friendly**: Git can track each package independently
### 🛡️ **Safety Features**
- **Dry-run mode**: Preview what would be done without making changes
- **Conflict resolution**: Handles existing files and directories gracefully
- **Rollback capability**: Easy to unstow (remove) configurations
- **Target verification**: Ensures target directory exists and is writable
## Command Examples
```bash
# Basic stow operations
cd ~/dotfiles
stow zsh # Deploy zsh configuration
stow emacs git # Deploy multiple packages
stow --target=/opt/local mypackage # Custom target directory
# Management operations
stow --delete zsh # Remove zsh symlinks (unstow)
stow --restow emacs # Restow (unstow then stow again)
stow --simulate zsh # Dry run - show what would happen
# Advanced usage
stow --verbose zsh # Verbose output
stow --no-folding emacs # Disable tree folding optimization
stow --ignore='\.DS_Store' zsh # Ignore certain files
```
## Directory Structure Example
```
~/dotfiles/ # Stow directory
├── zsh/ # Package: ZSH
│ ├── .zshrc # → ~/.zshrc
│ └── .config/ # → ~/.config/
│ └── zsh/ # → ~/.config/zsh/
│ └── aliases.zsh # → ~/.config/zsh/aliases.zsh
├── emacs/ # Package: Emacs
│ ├── .emacs.d/ # → ~/.emacs.d/
│ │ ├── init.el # → ~/.emacs.d/init.el
│ │ └── config.org # → ~/.emacs.d/config.org
│ └── .config/ # → ~/.config/
│ └── emacs/ # → ~/.config/emacs/
├── git/ # Package: Git
│ ├── .gitconfig # → ~/.gitconfig
│ └── .gitignore_global # → ~/.gitignore_global
└── nixos/ # Package: NixOS-specific
└── .config/ # → ~/.config/
└── nixos/ # → ~/.config/nixos/
└── user.nix # → ~/.config/nixos/user.nix
```
## Comparison with Current Org-Mode Approach
| Feature | Current Org-Mode | GNU Stow |
|---------|------------------|----------|
| **Configuration format** | Literate programming | Traditional dotfiles |
| **Deployment method** | Tangle + copy/symlink | Automatic symlinking |
| **Documentation** | Embedded in code | Separate documentation |
| **Version control** | Single org file | Multiple files in packages |
| **Learning curve** | Steep (Emacs/Org) | Gentle (simple concept) |
| **Flexibility** | Very high | Moderate |
| **Maintenance** | Manual tangling | Automatic linking |
| **Cross-platform** | Excellent | Excellent |
| **NixOS integration** | Native | External tool |
## Advantages of GNU Stow
### ✅ **Simplicity & Reliability**
- **Minimal learning curve**: Understand symlinks = understand Stow
- **No dependencies**: Works on any Unix-like system with Perl
- **Predictable behavior**: Simple, well-defined symlink creation rules
- **Mature and stable**: Decades of development and testing
### ✅ **Flexible Organization**
- **Package-based**: Logical separation of application configurations
- **Selective deployment**: Deploy only needed configs per machine
- **Easy experimentation**: Test new configs without affecting others
- **Conflict handling**: Safe deployment with conflict detection
### ✅ **Version Control Benefits**
- **Git-friendly**: Each package can be tracked separately
- **Branching**: Different branches for different machine types
- **History**: Track changes to individual application configs
- **Collaboration**: Easy to share specific application configurations
### ✅ **Multi-Machine Management**
```bash
# Different configs for different machine types
stow --target=/home/geir desktop # Desktop-specific configs
stow --target=/home/sma server # Server-specific configs
# Machine-specific packages
dotfiles/
├── desktop-geir/ # Only for desktop user geir
├── server-sma/ # Only for server user sma
└── common/ # Shared configurations
```
## Disadvantages & Limitations
### ❌ **Compared to Org-Mode Approach**
- **No literate programming**: Configuration and documentation separate
- **Less integration**: Not as tightly integrated with Emacs workflow
- **File-based**: No ability to generate configs from templates
- **Static configuration**: Less dynamic than org-mode tangling
### ❌ **General Limitations**
- **Symlink visibility**: Some applications don't handle symlinks well
- **Permission issues**: Target directory permissions must be correct
- **Perl dependency**: Requires Perl runtime (usually not an issue)
- **No config generation**: Can't generate configs based on system state
## Integration with Current NixOS Setup
### 🎯 **Hybrid Approach Options**
**Option 1: Stow for Traditional Dotfiles**
```bash
# Keep org-mode for complex Emacs config
# Use Stow for simple dotfiles
dotfiles/
├── emacs.org # Keep current literate approach
├── zsh/ # Simple dotfiles via Stow
│ └── .zshrc
└── git/
└── .gitconfig
```
**Option 2: User-Specific Deployment**
```bash
# Different approaches per user
~/dotfiles-geir/ # Org-mode for desktop user
~/dotfiles-sma/ # Stow for server user (simpler configs)
```
**Option 3: NixOS + Stow Integration**
```nix
# In NixOS configuration
environment.systemPackages = [ pkgs.stow ];
# User activation script
system.userActivationScripts.stow-dotfiles = ''
cd /home/geir/dotfiles
${pkgs.stow}/bin/stow --target=/home/geir common zsh git
'';
```
### 🔧 **Implementation Strategies**
**For Server Configurations (sma user):**
- Simple, minimal dotfiles via Stow
- Basic shell, git, and utility configurations
- No complex literate programming needed
**For Desktop Configurations (geir user):**
- Keep org-mode for Emacs (complex, well-integrated)
- Consider Stow for simpler applications (git, zsh aliases)
- Hybrid approach based on complexity
## Practical Examples
### 📝 **Basic Setup**
```bash
# Initialize dotfiles repository
mkdir ~/dotfiles
cd ~/dotfiles
# Create package structure
mkdir -p zsh git emacs
# Add configurations
echo 'alias ll="ls -la"' > zsh/.zshrc
echo '[user]
name = Geir Okkenhaug Jerstad
email = geir@geokkjer.eu' > git/.gitconfig
# Deploy configurations
stow zsh git
```
### 🏠 **Home Lab Specific Usage**
```bash
# Machine-specific dotfiles
dotfiles/
├── server-common/ # Shared server configs
│ ├── .zshrc
│ └── .vimrc
├── sleeper-service/ # NFS-specific configs
│ └── .config/
│ └── nfs/
└── grey-area/ # Git server specific
└── .gitconfig
# Deploy on sleeper-service
stow server-common sleeper-service
# Deploy on grey-area
stow server-common grey-area
```
### 🔄 **Migration Strategy**
```bash
# Phase 1: Extract simple configs from org-mode
emacs config.org
# Tangle simple configs to separate files
# Create Stow packages for extracted configs
# Phase 2: Test Stow deployment
stow --simulate zsh # Test before applying
stow zsh # Deploy if tests pass
# Phase 3: Maintain hybrid approach
# Keep org-mode for complex configs (Emacs)
# Use Stow for simple configs (shell, git)
```
## Integration with Current Workflow
### 🎯 **Recommended Approach for Home Lab**
**Keep Current Org-Mode For:**
- Emacs configuration (complex, well-integrated)
- Complex, documented configurations
- Configurations that benefit from literate programming
**Use GNU Stow For:**
- Server user (sma) configurations
- Simple shell configurations
- Git configurations
- Application-specific dotfiles that don't need documentation
**Implementation Plan:**
1. **Create stow repository** for server configs
2. **Test on sleeper-service** with sma user
3. **Gradually migrate** simple configs from org-mode
4. **Maintain hybrid approach** based on complexity needs
## Conclusion & Recommendation
### 🎯 **For Our Home Lab Context**
**GNU Stow is excellent for:**
- Server configurations (sma user)
- Simple, stable dotfiles
- Multi-machine deployments
- Configurations that don't need complex documentation
**Current org-mode approach excels for:**
- Complex Emacs configurations
- Documented, literate configurations
- Generated configurations
- Desktop user (geir) complex setups
### 📋 **Action Plan**
1. **Immediate**: Create stow-based dotfiles repository for server users
2. **Short-term**: Deploy simple sma user configs via Stow on sleeper-service
3. **Medium-term**: Evaluate migration of simple configs from org-mode
4. **Long-term**: Maintain hybrid approach - Stow for simple, org-mode for complex
**Verdict**: GNU Stow is an excellent complement to our current org-mode approach. Use Stow for simple, server-side configurations while keeping org-mode for complex desktop configurations. This provides the best of both worlds - simplicity where appropriate, power where needed.

View file

@ -1,298 +0,0 @@
# SSH Forwarding Solutions for Git Server
## Overview
This document researches solutions for forwarding SSH traffic from the reverse-proxy (VPS) to the grey-area Forgejo git instance. The goal is to enable Git operations over SSH at `git@git.geokkjer.eu` while maintaining security and reliability.
## Current Setup
- **Reverse Proxy (VPS)**: `46.226.104.98` running nginx, listening on ports 80/443
- **Git Server (grey-area)**: Forgejo instance accessible via Tailscale at `apps:3000`
- **Domain**: `git.geokkjer.eu` currently configured for HTTPS proxying only
- **Target**: Enable SSH access on port 22 for Git operations
## Architecture Constraints
1. **Single Public IP**: Only one public IP available (VPS)
2. **Port 22 Limitation**: Standard SSH port needed for Git compatibility
3. **Security**: SSH access should be restricted to Git operations only
4. **Tailscale Network**: Backend services accessible via private Tailscale network
5. **Existing SSH**: VPS already uses port 22 for administrative SSH access
## Solution Options
### 1. HAProxy TCP Mode (Recommended)
**Approach**: Replace nginx with HAProxy or add HAProxy for SSH forwarding
**Pros**:
- Native TCP/SSH forwarding support
- Battle-tested for SSH proxying
- Can handle both HTTP and SSH traffic
- Excellent connection handling
- Built-in health checking
**Cons**:
- Requires replacing or complementing nginx
- Additional service to manage
- More complex configuration
**Implementation**:
```haproxy
global
stats socket /var/run/haproxy.sock mode 600 level admin
stats timeout 2m
defaults
mode tcp
timeout client 60s
timeout server 60s
timeout connect 5s
# SSH forwarding to Git server
frontend ssh_frontend
bind *:22
mode tcp
# Use SNI or other method to distinguish Git SSH from admin SSH
default_backend git_ssh_backend
backend git_ssh_backend
mode tcp
server git1 apps:22 check
# HTTP/HTTPS forwarding
frontend http_frontend
bind *:80
bind *:443 ssl crt /etc/ssl/certs/
mode http
default_backend nginx_backend
backend nginx_backend
mode http
server nginx1 127.0.0.1:8080
```
**Complexity**: Medium
**Security**: High
**Reliability**: High
### 2. NGINX Stream Module
**Approach**: Enable NGINX stream module for TCP forwarding
**Pros**:
- Keeps existing nginx setup
- Native TCP proxying support
- Good performance
- Integrated with current SSL/certificate management
**Cons**:
- Requires nginx with stream module compiled
- Cannot distinguish SSH traffic types on same port
- Conflicts with existing SSH access
**Implementation**:
```nginx
# In nginx.conf
stream {
upstream git_ssh_backend {
server apps:22;
}
server {
listen 2222; # Alternative port for Git SSH
proxy_pass git_ssh_backend;
proxy_timeout 60s;
proxy_connect_timeout 5s;
}
}
http {
# Existing HTTP configuration
server {
listen 80;
listen 443 ssl;
server_name git.geokkjer.eu;
# ... existing config
}
}
```
**Complexity**: Low
**Security**: Medium (requires alternative port)
**Reliability**: High
### 3. SSH Port Forwarding with systemd
**Approach**: Use SSH tunneling with autossh/systemd for persistence
**Pros**:
- Simple setup
- Uses existing SSH infrastructure
- Automatic reconnection
- No additional services
**Cons**:
- Less robust than dedicated proxy
- Potential authentication complexity
- Single point of failure
- Not designed for high-throughput Git operations
**Implementation**:
```bash
# SSH tunnel command
ssh -N -L 2222:apps:22 geir@grey-area-tailscale-ip
# systemd service for persistence
[Unit]
Description=SSH tunnel for Git forwarding
After=network.target
[Service]
Type=simple
User=git
ExecStart=/usr/bin/ssh -N -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -L 2222:apps:22 geir@grey-area
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
```
**Complexity**: Low
**Security**: Medium
**Reliability**: Medium
### 4. iptables DNAT (Network Address Translation)
**Approach**: Use iptables to redirect SSH traffic to backend
**Pros**:
- Kernel-level forwarding (fast)
- Transparent to applications
- No additional services
- Can work on same port
**Cons**:
- Requires careful firewall management
- Complex routing setup with Tailscale
- Difficult to debug
- Less visibility into connections
**Implementation**:
```bash
# Forward port 2222 to backend Git server
iptables -t nat -A PREROUTING -p tcp --dport 2222 -j DNAT --to-destination apps:22
iptables -t nat -A POSTROUTING -p tcp -d apps --dport 22 -j MASQUERADE
# Allow forwarding
iptables -A FORWARD -p tcp --dport 22 -d apps -j ACCEPT
```
**Complexity**: High
**Security**: Medium
**Reliability**: Medium
### 5. Hybrid Solution: Port-Based Routing
**Approach**: Use different ports for different SSH services
**Pros**:
- Clear separation of concerns
- Maintains existing admin SSH on port 22
- Simple to implement and debug
- Good security boundaries
**Cons**:
- Non-standard Git SSH port
- Requires client configuration
- Less user-friendly
**Implementation**:
```nginx
stream {
upstream git_ssh {
server apps:22;
}
server {
listen 2222; # Git SSH port
proxy_pass git_ssh;
proxy_timeout 300s;
proxy_connect_timeout 10s;
}
}
```
**Client Configuration**:
```bash
# ~/.ssh/config
Host git.geokkjer.eu
Port 2222
User git
```
**Complexity**: Low
**Security**: High
**Reliability**: High
## Recommended Solution
### Phase 1: Implement NGINX Stream Module with Alternative Port
**Why**:
- Lowest risk implementation
- Maintains existing services
- Easy to test and validate
- Can be upgraded later
**Configuration**:
1. Enable nginx stream module
2. Configure Git SSH on port 2222
3. Update Forgejo SSH_DOMAIN setting
4. Test with alternative port
### Phase 2: Consider HAProxy Migration (Future)
**Why**:
- More robust for mixed TCP/HTTP traffic
- Better connection handling for Git operations
- Industry standard for this use case
- Enables port 22 for Git SSH
## Security Considerations
1. **SSH Key Management**: Ensure proper SSH key authentication
2. **Rate Limiting**: Implement connection rate limits
3. **Monitoring**: Log SSH connections for security auditing
4. **Firewall Rules**: Restrict SSH forwarding to specific sources if possible
5. **Fail2ban**: Extend fail2ban rules to cover Git SSH attempts
## Testing Strategy
1. **Basic Connectivity**: Test TCP connection to backend
2. **SSH Handshake**: Verify SSH protocol negotiation
3. **Git Operations**: Test clone, push, pull operations
4. **Performance**: Measure latency and throughput
5. **Failover**: Test behavior during backend unavailability
## Implementation Priority
1. **High Priority**: NGINX Stream Module (Phase 1)
2. **Medium Priority**: HAProxy migration evaluation
3. **Low Priority**: Advanced features (rate limiting, monitoring)
## Next Steps
1. Check if nginx has stream module compiled
2. Implement NGINX stream configuration for port 2222
3. Update Forgejo SSH configuration
4. Test Git operations via new port
5. Document client configuration for users
6. Monitor performance and reliability
## References
- [NGINX Stream Module Documentation](http://nginx.org/en/docs/stream/ngx_stream_core_module.html)
- [HAProxy TCP Mode Configuration](https://www.haproxy.org/download/2.8/doc/configuration.txt)
- [Git SSH Configuration Best Practices](https://docs.github.com/en/authentication/connecting-to-github-with-ssh)

0
users/README.md Normal file
View file