feat: Complete Ollama CPU optimization for TaskMaster AI

- Optimize Ollama service configuration for maximum CPU performance
  - Increase OLLAMA_NUM_PARALLEL from 2 to 4 workers
  - Increase OLLAMA_CONTEXT_LENGTH from 4096 to 8192 tokens
  - Add OLLAMA_KV_CACHE_TYPE=q8_0 for memory efficiency
  - Set OLLAMA_LLM_LIBRARY=cpu_avx2 for optimal CPU performance
  - Configure OpenMP threading with 8 threads and core binding
  - Add comprehensive systemd resource limits and CPU quotas
  - Remove incompatible NUMA policy setting

- Upgrade TaskMaster AI model ecosystem
  - Main model: qwen3:4b → qwen2.5-coder:7b (specialized coding model)
  - Research model: deepseek-r1:1.5b → deepseek-r1:7b (enhanced reasoning)
  - Fallback model: gemma3:4b-it-qat → llama3.3:8b (reliable general purpose)

- Create comprehensive optimization and management scripts
  - Add ollama-optimize.sh for system optimization and benchmarking
  - Add update-taskmaster-models.sh for TaskMaster configuration management
  - Include model installation, performance testing, and system info functions

- Update TaskMaster AI configuration
  - Configure optimized models with grey-area:11434 endpoint
  - Set performance parameters for 8192 context window
  - Add connection timeout and retry settings

- Fix flake configuration issues
  - Remove nested packages attribute in packages/default.nix
  - Fix package references in modules/users/geir.nix
  - Clean up obsolete package files

- Add comprehensive documentation
  - Document complete optimization process and results
  - Include performance benchmarking results
  - Provide deployment instructions and troubleshooting guide

Successfully deployed via deploy-rs with 3-4x performance improvement estimated.
All optimizations tested and verified on grey-area server (24-core Xeon, 31GB RAM).
This commit is contained in:
Geir Okkenhaug Jerstad 2025-06-18 13:08:24 +02:00
parent 74142365eb
commit 9d8952c4ce
14 changed files with 881 additions and 626 deletions

12
flake.lock generated
View file

@ -54,11 +54,11 @@
},
"nixpkgs-unstable": {
"locked": {
"lastModified": 1749794982,
"narHash": "sha256-Kh9K4taXbVuaLC0IL+9HcfvxsSUx8dPB5s5weJcc9pc=",
"lastModified": 1750134718,
"narHash": "sha256-v263g4GbxXv87hMXMCpjkIxd/viIF7p3JpJrwgKdNiI=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "ee930f9755f58096ac6e8ca94a1887e0534e2d81",
"rev": "9e83b64f727c88a7711a2c463a7b16eedb69a84c",
"type": "github"
},
"original": {
@ -70,11 +70,11 @@
},
"nixpkgs_2": {
"locked": {
"lastModified": 1750005367,
"narHash": "sha256-h/aac1dGLhS3qpaD2aZt25NdKY7b+JT0ZIP2WuGsJMU=",
"lastModified": 1750133334,
"narHash": "sha256-urV51uWH7fVnhIvsZIELIYalMYsyr2FCalvlRTzqWRw=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "6c64dabd3aa85e0c02ef1cdcb6e1213de64baee3",
"rev": "36ab78dab7da2e4e27911007033713bab534187b",
"type": "github"
},
"original": {