home-lab/documentation/OLLAMA_DEPLOYMENT_SUMMARY.md

Ollama Service Deployment Summary

What Was Created

I've researched and implemented a comprehensive Ollama service configuration for your NixOS home lab. Here's what's been added:

1. Research Documentation

• /home/geir/Home-lab/research/ollama.md - Comprehensive research on Ollama, including features, requirements, security considerations, and deployment recommendations.

2. NixOS Module

• /home/geir/Home-lab/modules/services/ollama.nix - A complete NixOS module for Ollama with:
  • Secure service isolation
  • Configurable network binding
  • Resource management
  • GPU acceleration support
  • Health monitoring
  • Automatic model downloads
  • Backup functionality

3. Service Configuration

• /home/geir/Home-lab/machines/grey-area/services/ollama.nix - Specific configuration for deploying Ollama on grey-area with:
  • 3 popular models (llama3.3:8b, codellama:7b, mistral:7b)
  • Resource limits to protect other services
  • Security-focused localhost binding
  • Monitoring and health checks enabled

4. Management Tools

• /home/geir/Home-lab/scripts/ollama-cli.sh - CLI tool for common Ollama operations
• /home/geir/Home-lab/scripts/monitor-ollama.sh - Comprehensive monitoring script

5. Documentation

• /home/geir/Home-lab/documentation/OLLAMA_DEPLOYMENT.md - Complete deployment guide
• /home/geir/Home-lab/documentation/OLLAMA_INTEGRATION_EXAMPLES.md - Integration examples for development workflow

6. Configuration Updates

• Updated grey-area/configuration.nix to include the Ollama service
• Enhanced home-lab-tools package with Ollama tool references

Quick Deployment

To deploy Ollama to your grey-area server:

# Navigate to your home lab directory
cd /home/geir/Home-lab

# Deploy the updated configuration
sudo nixos-rebuild switch --flake .#grey-area

What Happens During Deployment

1. Service Creation: Ollama systemd service will be created and started
2. User/Group Setup: Dedicated ollama user and group created for security
3. Model Downloads: Three AI models will be automatically downloaded:
   • llama3.3:8b (~4.7GB) - General purpose model
   • codellama:7b (~3.8GB) - Code-focused model
   • mistral:7b (~4.1GB) - Fast inference model
4. Directory Setup: /var/lib/ollama created for model storage
5. Security Hardening: Service runs with restricted permissions
6. Resource Limits: Memory limited to 12GB, CPU to 75%

Post-Deployment Verification

After deployment, verify everything is working:

# Check service status
systemctl status ollama

# Test API connectivity
curl http://localhost:11434/api/tags

# Use the CLI tool
/home/geir/Home-lab/scripts/ollama-cli.sh status

# Run comprehensive monitoring
/home/geir/Home-lab/scripts/monitor-ollama.sh --test-inference

Storage Requirements

The initial setup will download approximately 12.6GB of model data:

• llama3.3:8b: ~4.7GB
• codellama:7b: ~3.8GB
• mistral:7b: ~4.1GB

Ensure grey-area has sufficient storage space.

Usage Examples

Once deployed, you can use Ollama for:

Interactive Chat

# Start interactive session with a model
ollama run llama3.3:8b

# Code assistance
ollama run codellama:7b "Review this function for security issues"

API Usage

# Generate text via API
curl -X POST http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{"model": "llama3.3:8b", "prompt": "Explain NixOS modules", "stream": false}'

# OpenAI-compatible API
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "mistral:7b", "messages": [{"role": "user", "content": "Hello!"}]}'

CLI Tool

# Using the provided CLI tool
ollama-cli.sh models          # List installed models
ollama-cli.sh chat mistral:7b # Start chat session
ollama-cli.sh test            # Run functionality tests
ollama-cli.sh pull phi4:14b   # Install additional models

Security Configuration

The deployment uses secure defaults:

• Network Binding: localhost only (127.0.0.1:11434)
• User Isolation: Dedicated ollama user with minimal permissions
• Systemd Hardening: Extensive security restrictions applied
• No External Access: Firewall closed by default

To enable external access, consider using a reverse proxy (examples provided in documentation).

Resource Management

The service includes resource limits to prevent impact on other grey-area services:

• Memory Limit: 12GB maximum
• CPU Limit: 75% maximum
• Process Isolation: Separate user and group
• File System Restrictions: Limited write access

Monitoring and Maintenance

The deployment includes:

• Health Checks: Automated service health monitoring
• Backup System: Configuration and custom model backup
• Log Management: Structured logging with rotation
• Performance Monitoring: Resource usage tracking

Next Steps

1. Deploy: Run the nixos-rebuild command above
2. Verify: Check service status and API connectivity
3. Test: Try the CLI tools and API examples
4. Integrate: Use the integration examples for your development workflow
5. Monitor: Set up regular monitoring using the provided tools

Troubleshooting

If you encounter issues:

1. Check Service Status: systemctl status ollama
2. View Logs: journalctl -u ollama -f
3. Monitor Downloads: journalctl -u ollama-model-download -f
4. Run Diagnostics: /home/geir/Home-lab/scripts/monitor-ollama.sh
5. Check Storage: df -h /var/lib/ollama

Future Enhancements

Consider these potential improvements:

• GPU Acceleration: Enable if you add a compatible GPU to grey-area
• Web Interface: Deploy Open WebUI for browser-based interaction
• External Access: Configure reverse proxy for remote access
• Additional Models: Install specialized models for specific tasks
• Integration: Implement the development workflow examples

The Ollama service is now ready to provide local AI capabilities to your home lab infrastructure!