381 lines
10 KiB
Markdown
381 lines
10 KiB
Markdown
# NFS Best Practices and Troubleshooting Guide
|
|
|
|
## Overview
|
|
|
|
Network File System (NFS) is a distributed file system protocol that allows remote file access over a network. This document outlines best practices for configuring NFS in a home lab environment, focusing on permission management, security, and performance optimization.
|
|
|
|
## Current Configuration Analysis
|
|
|
|
### Identified Issues
|
|
|
|
1. **Missing User ID Mapping**: The current configuration doesn't ensure consistent UIDs/GIDs between server and clients
|
|
2. **Security Concerns**: Using `no_root_squash` poses security risks
|
|
3. **No ID Mapping Configuration**: NFSv4 ID mapping is not configured
|
|
4. **Missing Export Options**: Several important security and performance options are not set
|
|
|
|
## NFS Permission Management Best Practices
|
|
|
|
### 1. User and Group ID Consistency
|
|
|
|
**Problem**: NFS relies on UID/GID matching between server and clients. If a user has UID 1000 on the server but UID 1001 on the client, permission issues will occur.
|
|
|
|
**Solutions**:
|
|
|
|
#### Option A: Synchronized UIDs/GIDs (Recommended for Home Lab)
|
|
- Ensure the `media` group has the same GID (993) on all machines
|
|
- Create users with consistent UIDs across all systems
|
|
- Use centralized user management (LDAP/AD) for larger setups
|
|
|
|
#### Option B: NFSv4 ID Mapping
|
|
Configure NFSv4 ID mapping to translate between different UID/GID spaces:
|
|
|
|
```nix
|
|
# On both server and clients
|
|
services.rpcbind.enable = true;
|
|
services.nfs.idmapd = {
|
|
enable = true;
|
|
settings = {
|
|
General = {
|
|
Domain = "home.lab"; # Same domain on all machines
|
|
Verbosity = 0;
|
|
};
|
|
Mapping = {
|
|
Nobody-User = "nobody";
|
|
Nobody-Group = "nogroup";
|
|
};
|
|
};
|
|
};
|
|
```
|
|
|
|
#### Option C: Use all_squash for Public Shares
|
|
For truly shared directories where ownership doesn't matter:
|
|
|
|
```nix
|
|
exports = ''
|
|
/mnt/storage/shares 10.0.0.0/24(rw,sync,all_squash,anonuid=993,anongid=993)
|
|
'';
|
|
```
|
|
|
|
### 2. Security Best Practices
|
|
|
|
#### Root Squashing (Critical)
|
|
**Current Issue**: Using `no_root_squash` allows root on clients to access files as root on the server.
|
|
|
|
**Fix**: Use root squashing by default and only disable when absolutely necessary:
|
|
|
|
```nix
|
|
exports = ''
|
|
# Default: root_squash is enabled (secure)
|
|
/mnt/storage/media 10.0.0.0/24(rw,sync,no_subtree_check,root_squash)
|
|
|
|
# Only for trusted admin workstations
|
|
/mnt/storage/backups 10.0.0.100(rw,sync,no_subtree_check,no_root_squash)
|
|
'';
|
|
```
|
|
|
|
#### Port Security
|
|
Always use the `secure` option (enabled by default) to restrict access to privileged ports:
|
|
|
|
```nix
|
|
exports = ''
|
|
/mnt/storage/media 10.0.0.0/24(rw,sync,secure,no_subtree_check)
|
|
'';
|
|
```
|
|
|
|
#### Client Mount Security
|
|
On client systems, use security options:
|
|
|
|
```bash
|
|
# Mount with nosuid to prevent privilege escalation
|
|
mount -t nfs -o nosuid,nodev server:/mnt/storage/media /mnt/media
|
|
|
|
# In /etc/fstab
|
|
server:/mnt/storage/media /mnt/media nfs nosuid,nodev,rw,hard,timeo=600 0 0
|
|
```
|
|
|
|
### 3. Export Options Explained
|
|
|
|
#### Permission and Security Options
|
|
- `root_squash`: Map root UID to anonymous user (default, secure)
|
|
- `no_root_squash`: Allow root access (use carefully)
|
|
- `all_squash`: Map all users to anonymous user
|
|
- `anonuid=N`: Set anonymous user UID
|
|
- `anongid=N`: Set anonymous group GID
|
|
|
|
#### Performance Options
|
|
- `sync`: Writes are committed before responding (safer, slower)
|
|
- `async`: Writes may be cached (faster, less safe)
|
|
- `no_wdelay`: Don't delay writes (for small random writes)
|
|
|
|
#### Access Control Options
|
|
- `rw`: Read-write access
|
|
- `ro`: Read-only access
|
|
- `secure`: Only accept requests from privileged ports (default)
|
|
- `insecure`: Accept requests from any port
|
|
|
|
#### Filesystem Options
|
|
- `no_subtree_check`: Don't verify file is in exported subtree (recommended)
|
|
- `subtree_check`: Verify file location (slight security benefit, performance cost)
|
|
|
|
## Recommended NFS Configuration
|
|
|
|
### Server Configuration (nfs.nix)
|
|
|
|
```nix
|
|
{
|
|
config,
|
|
pkgs,
|
|
...
|
|
}: {
|
|
imports = [
|
|
../../modules/users/media-group.nix
|
|
];
|
|
|
|
# NFSv4 ID mapping
|
|
services.rpcbind.enable = true;
|
|
services.nfs.idmapd = {
|
|
enable = true;
|
|
settings = {
|
|
General = {
|
|
Domain = "home.lab";
|
|
Verbosity = 0;
|
|
};
|
|
Mapping = {
|
|
Nobody-User = "nobody";
|
|
Nobody-Group = "nogroup";
|
|
};
|
|
};
|
|
};
|
|
|
|
# NFS server configuration
|
|
services.nfs.server = {
|
|
enable = true;
|
|
# Optimized exports with proper security
|
|
exports = ''
|
|
# Main storage - root squashed for security
|
|
/mnt/storage 10.0.0.0/24(rw,sync,no_subtree_check,crossmnt) 100.64.0.0/10(rw,sync,no_subtree_check,crossmnt)
|
|
|
|
# Media directory - accessible to media group
|
|
/mnt/storage/media 10.0.0.0/24(rw,sync,no_subtree_check,root_squash) 100.64.0.0/10(rw,sync,no_subtree_check,root_squash)
|
|
|
|
# Downloads - squash all users to media group
|
|
/mnt/storage/downloads 10.0.0.0/24(rw,sync,no_subtree_check,all_squash,anonuid=993,anongid=993) 100.64.0.0/10(rw,sync,no_subtree_check,all_squash,anonuid=993,anongid=993)
|
|
|
|
# Backups - admin only with root access
|
|
/mnt/storage/backups 10.0.0.100(rw,sync,no_subtree_check,no_root_squash)
|
|
|
|
# Public shares - anonymous access
|
|
/mnt/storage/shares 10.0.0.0/24(rw,sync,no_subtree_check,all_squash,anonuid=993,anongid=993) 100.64.0.0/10(ro,sync,no_subtree_check,all_squash)
|
|
'';
|
|
createMountPoints = true;
|
|
};
|
|
|
|
# Directory permissions and ownership
|
|
systemd.tmpfiles.rules = [
|
|
# Media group directories
|
|
"d /mnt/storage/media 2775 root media -" # Setgid for group inheritance
|
|
"d /mnt/storage/downloads 2775 media media -" # Owned by media group
|
|
"d /mnt/storage/backups 0750 root root -" # Admin only
|
|
"d /mnt/storage/shares 2775 media media -" # Public access via media group
|
|
];
|
|
|
|
# Performance tuning
|
|
boot.kernel.sysctl = {
|
|
# Increase NFS server thread count for better performance
|
|
"fs.nfs.nlm_tcpport" = 32768;
|
|
"fs.nfs.nlm_udpport" = 32768;
|
|
};
|
|
|
|
# Required packages
|
|
environment.systemPackages = with pkgs; [
|
|
nfs-utils
|
|
];
|
|
|
|
# Firewall configuration
|
|
networking.firewall = {
|
|
allowedTCPPorts = [
|
|
111 # portmapper
|
|
2049 # nfsd
|
|
32768 # lockd
|
|
];
|
|
allowedUDPPorts = [
|
|
111 # portmapper
|
|
2049 # nfsd
|
|
32768 # lockd
|
|
];
|
|
};
|
|
}
|
|
```
|
|
|
|
### Client Configuration
|
|
|
|
```nix
|
|
# Enable NFS client services
|
|
services.rpcbind.enable = true;
|
|
services.nfs.idmapd = {
|
|
enable = true;
|
|
settings = {
|
|
General = {
|
|
Domain = "home.lab"; # Must match server
|
|
};
|
|
};
|
|
};
|
|
|
|
# Example mount in /etc/fstab
|
|
# server:/mnt/storage/media /mnt/media nfs4 rw,hard,timeo=600,retrans=5,_netdev,nosuid 0 0
|
|
```
|
|
|
|
## Troubleshooting Common Permission Issues
|
|
|
|
### 1. "Permission Denied" Errors
|
|
|
|
**Symptoms**: Users cannot access files they should be able to access
|
|
|
|
**Diagnosis**:
|
|
```bash
|
|
# Check UID/GID mapping
|
|
id username # On both client and server
|
|
|
|
# Check export configuration
|
|
exportfs -v
|
|
|
|
# Check mount options
|
|
mount | grep nfs
|
|
|
|
# Check file permissions
|
|
ls -la /mnt/storage/media
|
|
```
|
|
|
|
**Solutions**:
|
|
- Ensure UID/GID consistency
|
|
- Check export options (root_squash, all_squash)
|
|
- Verify group membership
|
|
- Check directory setgid bit (2xxx permissions)
|
|
|
|
### 2. "Operation Not Permitted" for File Operations
|
|
|
|
**Symptoms**: Can read files but cannot create/modify/delete
|
|
|
|
**Solutions**:
|
|
- Check write permissions in exports (`rw` vs `ro`)
|
|
- Verify directory permissions (need write + execute)
|
|
- Check if filesystem is mounted read-only
|
|
- Ensure no conflicting mount options (`ro`, `nosuid`)
|
|
|
|
### 3. Files Created with Wrong Ownership
|
|
|
|
**Symptoms**: New files appear with unexpected UID/GID
|
|
|
|
**Solutions**:
|
|
- Use setgid bit on directories (`chmod g+s /directory`)
|
|
- Configure all_squash with appropriate anonuid/anongid
|
|
- Set up proper ID mapping
|
|
- Use ACLs for complex permission scenarios
|
|
|
|
### 4. Root Access Issues
|
|
|
|
**Symptoms**: Root operations fail on NFS mounts
|
|
|
|
**Solutions**:
|
|
- Check if `root_squash` is intended behavior
|
|
- Use `no_root_squash` only for trusted admin clients
|
|
- Consider using `sudo` on the NFS server instead
|
|
|
|
## Performance Optimization
|
|
|
|
### Server Tuning
|
|
|
|
```nix
|
|
# Increase NFS daemon threads
|
|
services.nfs.server.threads = 16;
|
|
|
|
# Kernel parameters for better NFS performance
|
|
boot.kernel.sysctl = {
|
|
"net.core.rmem_max" = 134217728;
|
|
"net.core.wmem_max" = 134217728;
|
|
"net.ipv4.tcp_rmem" = "4096 65536 134217728";
|
|
"net.ipv4.tcp_wmem" = "4096 65536 134217728";
|
|
};
|
|
```
|
|
|
|
### Client Tuning
|
|
|
|
```bash
|
|
# Mount with performance options
|
|
mount -t nfs4 -o rsize=1048576,wsize=1048576,hard,timeo=600 server:/path /mount
|
|
```
|
|
|
|
## Security Considerations
|
|
|
|
### 1. Network Security
|
|
- Use VPN or firewall rules to restrict NFS access
|
|
- Consider NFSv4 with Kerberos for authentication
|
|
- Enable RPC-with-TLS for encryption (Linux 6.5+)
|
|
|
|
### 2. File System Security
|
|
- Use minimal necessary permissions
|
|
- Regular security audits of export configurations
|
|
- Monitor NFS access logs
|
|
- Implement backup and recovery procedures
|
|
|
|
### 3. Access Control
|
|
```nix
|
|
# Example of layered security approach
|
|
exports = ''
|
|
# Development - developers only, root squashed
|
|
/srv/dev 192.168.1.0/24(rw,sync,root_squash,no_subtree_check)
|
|
|
|
# Public - read-only, all users squashed
|
|
/srv/public *(ro,sync,all_squash,no_subtree_check)
|
|
|
|
# Admin - restricted to specific hosts, root access
|
|
/srv/admin 192.168.1.10(rw,sync,no_root_squash,no_subtree_check)
|
|
'';
|
|
```
|
|
|
|
## Monitoring and Maintenance
|
|
|
|
### Server Monitoring
|
|
```bash
|
|
# Check NFS statistics
|
|
cat /proc/net/rpc/nfsd
|
|
|
|
# Monitor NFS threads
|
|
cat /proc/fs/nfsd/threads
|
|
|
|
# Check exports
|
|
exportfs -v
|
|
|
|
# Monitor client connections
|
|
ss -tuln | grep :2049
|
|
```
|
|
|
|
### Client Monitoring
|
|
```bash
|
|
# Check mount status
|
|
mount | grep nfs
|
|
|
|
# Monitor NFS statistics
|
|
nfsstat -c
|
|
|
|
# Check for stale handles
|
|
dmesg | grep -i nfs
|
|
```
|
|
|
|
## References
|
|
|
|
- [Linux NFS-HOWTO](https://tldp.org/HOWTO/NFS-HOWTO/)
|
|
- [exports(5) Manual Page](https://man7.org/linux/man-pages/man5/exports.5.html)
|
|
- [Arch Linux NFS Wiki](https://wiki.archlinux.org/title/NFS)
|
|
- [NFS Performance Tuning](https://nfs.sourceforge.net/)
|
|
- [Red Hat NFS Documentation](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/managing_file_systems/)
|
|
|
|
## Conclusion
|
|
|
|
Proper NFS configuration requires careful attention to:
|
|
1. User/group ID consistency
|
|
2. Appropriate security settings
|
|
3. Performance optimization
|
|
4. Regular monitoring and maintenance
|
|
|
|
The recommended configuration provides a good balance of security, performance, and usability for a home lab environment. Always test changes in a development environment before applying to production systems.
|