HARDWARE PROFILE
Unraid 128 GB + RTX 3060
- CPU
- Intel Core i5-12600KF
- Memory
- 128 GB DDR4-3200
- GPU
- NVIDIA RTX 3060 12 GB
MEMORY BUDGET
Fit the whole system, not only the weights.
Keep total weights and runtime overhead below available host RAM; leave headroom for the OS and shared services.
BOTTLENECK
Bandwidth sets the ceiling.
Single-stream Flash-Next decode is limited by DDR4 memory bandwidth, not idle GPU compute.
PUBLIC-SAFE NOTES
- CPU-offloaded MoE inference
- Approximately 6.3 GiB VRAM used by the Q3_K_XL configuration
- Model weights stream from host memory
3 published runs use this profile.