- Nix 62.5%
- Shell 29.9%
- Python 7.6%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
Geir Okkenhaug Jerstad
a33a6818c9
Traefik logged at startup:
"SafeNaming is not explicitly set: generated routers, middlewares and services will
use the legacy naming scheme, under which names can collide across namespaces or
resources."
Set providers.kubernetesCRD.safeNaming: true. Upstream notes the change CAN be
breaking, so it is cheapest now — before any route depends on the generated names.
⚠️ Verified against the running binary (`traefik --help`) that this flag exists ONLY
for the CRD provider in v3.7.13. The kubernetesIngress and kubernetesGateway variants
do not exist, and passing one would have stopped Traefik from starting.
Re-verified after the change by running the RUNBOOK §6.7 procedure exactly as written,
with the manifest extracted from the doc itself: Ingress=served-by=INGRESS,
HTTPRoute=served-by=HTTPROUTE, reproducible 3x, unknown host 404, HTTPRoute
Accepted=True/ResolvedRefs=True, Ingress ADDRESS=10.0.0.240, warning count 0.
|
||
| .agents/notes | ||
| .github/PULL_REQUEST_TEMPLATE | ||
| assets | ||
| docs | ||
| fleet | ||
| k8s | ||
| plans | ||
| research | ||
| scripts | ||
| .envrc | ||
| .gitignore | ||
| AGENTS.md | ||
| LICENSE | ||
| README.md | ||
home-lab
A home lab in the middle of a rebuild: one Kubernetes cluster on Talos Linux, plus a fleet of NixOS machines that are not cluster nodes.
Cold start? Read this file, then docs/README.md for the reading order.
The four machines
| Host | Hardware | OS | What it does |
|---|---|---|---|
congenital-optimist |
Threadripper 1920X · 64 GB | Talos v1.14.1 | The cluster. Control plane + worker, 10.0.0.200 |
grey-area |
Xeon E5-2670 v3 · 32 GB | NixOS → becomes Talos | Currently runs Forgejo; becomes a cluster worker |
limiting-factor |
Intel N150 · 11 GB · 2×1.9 TB btrfs RAID1 | NixOS (permanent) | Storage, NFS, AdGuard DNS. 10.0.0.24 |
reverse-proxy |
VPS · 1 vCPU · 1.95 GB | NixOS (permanent) | Edge, TLS termination, Headscale. Not on the LAN |
Host names are Iain M. Banks Culture ship names — see docs/NAMING.md.
Network: LAN 10.0.0.0/24, gateway 10.0.0.138. Statics .200/.201; MetalLB pool .240–.249.
Overlay is Headscale (self-hosted, not Tailscale SaaS) at ts.geokkjer.eu. TLS terminates at the
VPS edge — there is deliberately no cert-manager.
The layout, and why it is shaped this way
Split by what a thing is, not by when it was written:
k8s/ THE CLUSTER — everything Kubernetes and Talos
talos/ machine configs: schematic + patches
bootstrap/ the one hand-applied layer (Argo CD, secrets, seed app)
gitops/ Argo CD Applications — the app-of-apps
fleet/ THE MACHINES — everything NixOS
flake.nix builds the hosts below, and this repo's devShell
hive.nix the fleet declared ONCE — nixosConfigurations and Colmena both derive from it
hosts/ one directory per live host
modules/ modules the live hosts actually import
workstation-modules/ Phase 8 material — laptop & desktop, not yet wired up
docs/ CROSS-CUTTING reference — runbook, lessons, network, hardware
plans/ the canonical plan: plans/HOME_LAB_REBUILD_2026.md
research/ 15 dated, cited reports — the evidence behind the decisions
scripts/ tooling; validate-all.sh is the one to run
.agents/ decision records (see AGENTS.md for the process)
k8s/ vs fleet/ because the old machines/ directory became a lie. It held NixOS configs for
hosts that are Talos now. A directory called machines/ cannot tell you which kind of machine you are
looking at; fleet/hosts/ can.
The flake lives in fleet/ because it builds the fleet. It also provides the repo's devShell
(talosctl, kubectl, helm, kustomize), which is why .envrc says use flake ./fleet — running
nix develop from the repo root still gives you the toolchain.
Where things stand
| Phase | What | Status |
|---|---|---|
| 0 | Backup and preserve | ✅ complete — Forgejo dump verified, media array scrubbed clean |
| 1 | Network and DNS | 🟡 Headscale live — congenital-optimist is registered at 100.64.0.1 (2026-10-07); AdGuard live but the router has not been cut over |
| 2 | The cluster | 🟡 control plane live 2026-10-06 — checkpoint NOT met — MetalLB is live and the VIP 10.0.0.240 answers from the LAN (2026-10-07), but the checkpoint still needs two nodes Ready, a fully rehearsed etcd restore, and grey-area wiped |
| 3 | Storage and ingress | 🟡 local-path and nfs-client both verified — a test PVC binds and is writable. Traefik v3.7.13 + Gateway API v1.6.2 live, a test app reachable via both an Ingress and an HTTPRoute (2026-10-07); static /media PVs and secrets not started |
| 4–7 | GitOps, delivery, services | Not started |
| 8 | Workstations → NixOS | Not started; material is staged in fleet/workstation-modules/ |
The full plan, with all 26 decisions and their trade-offs, is
plans/HOME_LAB_REBUILD_2026.md.
Do this first
direnv allow # or: nix develop ./fleet
./scripts/validate-all.sh # 9 checks — add --full for NixOS evaluation
Talking to the cluster:
# kubectl and k9s work directly — the kubeconfig is merged into ~/.kube/config
kubectl get nodes
k9s
# Talos itself
talosctl --talosconfig ./talosconfig -e 10.0.0.200 -n 10.0.0.200 version
Two things that will bite you
1. NVMe device names are not stable across reboots. They swapped on congenital-optimist between
two boots. Everything selects disks by serial, never by device name or type: nvme — doing otherwise
would have wiped the data disk. Before any disk operation, resolve the device from its serial:
talosctl --talosconfig ./talosconfig -e 10.0.0.200 -n 10.0.0.200 get disks -o yaml
2. Rotating the cluster CA requires physical access. There is no remote path: --insecure needs
maintenance mode, and authenticated access needs the CA you just replaced. Budget for a USB boot.
Known blockers
| Blocker | Detail |
|---|---|
| Router DNS cutover | AdGuard on 10.0.0.24 is live but the LAN still resolves via the Chromebox at 10.0.0.14. Do not unplug the Chromebox until the router points at 10.0.0.24. |
| Tailscale extension caused a reboot loop | An unconfigured ext-tailscale made Talos's boot sequence miss its ~70-minute deadline, so the node rebooted every 70 min 54 s (17 times). ✅ Fixed — config applied live and committed to both Talos patches; the node is now registered on the tailnet at 100.64.0.1. |
| Forgejo's future is undecided | D10 in the plan is marked UNDER REVISION — a proposal to replace Forgejo with a self-hosted forge is open in .agents/notes/proposed/. |
grey-area's Talos patch is unfilled |
It has never been inspected in Talos, so its disk serial and interface name are still placeholders in k8s/talos/patches/worker.yaml. |
Reading order
Start with docs/README.md. If you are an agent, read
AGENTS.md first — it carries the rules this project learned the hard way.