AI
Local inference and AI infrastructure.
-
Ollama Not Using Your NVIDIA GPU on Linux: What to Check
Check NVIDIA device visibility, Ollama model placement, systemd settings, and Docker GPU access when inference appears to run on CPU.
-
How to Estimate GPU VRAM for Local LLM Inference
Estimate local LLM VRAM from model weights, KV cache, context length, and concurrency, then check the estimate against the intended workload.