Local LLM
Running and understanding local language models.
-
[Local LLM 23] Check OpenClaw’s Ollama Endpoint and Tool Calls
Verify the Ollama endpoint used by OpenClaw, then distinguish connection, model selection and tool-calling problems.
-
[Local LLM 24] Use OpenRouter and Check Cost and Data Routing
Call OpenRouter through its API and inspect model names, pricing and the provider path before sending application data.
-
[Local LLM 25] Decide Which Requests Stay Local or Use the Cloud
Choose a local or cloud path based on data handling, quality, latency and cost, then record how failures and fallback are handled.
-
[Local LLM 26] Measure Loading, First Response and Generation Speed
Measure model loading, time to first response and token generation separately, using the same request and recorded runtime settings.
-
[Local LLM 27] Compare GGUF and CPU/GPU Options in llama.cpp
Run a GGUF model with llama.cpp and compare CPU and GPU settings while keeping the model and request conditions recorded.
-
[Local LLM 28] Understand LoRA, QLoRA and a Small Training Script
Separate inference from training, inspect LoRA and QLoRA requirements, and read the structure of a small adaptation script.
-
[Local LLM 29] Review API, Log and Tool Permissions
Check exposed APIs, credentials, logs, document access and agent tool permissions before sharing a local LLM service.
-
[Local LLM 30] Back Up and Restore a Document Q&A Tool
Keep document data, configuration and rebuild steps together, then test restoring the Q&A tool in a separate working directory.
-
Ollama Not Using Your NVIDIA GPU on Linux: What to Check
Check NVIDIA device visibility, Ollama model placement, systemd settings, and Docker GPU access when inference appears to run on CPU.