Building a Local LLM · 01/30 · Introduction
Full series and learning sequence · Practice code ZIP
Start by running one small model on the computer you already have. Give it a short note and check whether its summary preserves the important information. The series grows from that first response into document search, application integration and ongoing operation.
Prepare your Windows 11 PC and write down the memory and graphics card specifications. You don’t need to buy new equipment or install development tools yet.
Role of model, runtime, and screen
It is easier to understand if the composition is divided into three parts. LLM is a model that calculates the probability of the next token based on the previous context, and Ollama or llama.cpp is the runtime that executes the model. The chat window is a screen that sends requests at runtime and shows the answers. This is why receiving the model file and installing the chat screen are two different things.
| parts | Examples from this series | What changes when you change |
|---|---|---|
| model weights | Qwen·Gemma·Hermes | Language ability, size, conditions of use |
| Inference Runtime | Ollama·llama.cpp | Hardware support, API, execution options |
| Use screen | Basic app·LM Studio·Open WebUI | Conversation Management and Approach |
| agent | Hermes Agent·OpenClaw | Use tools, remember, and execute tasks |
You need to check separately where the app is installed and where the calculations are made. Apps installed on a PC can also send requests to an external API, and the browser screen can also connect to the model server on the same PC. Look for the model name, along with the server address and whether you are using a web search or external tool.
Check your PC specifications
Open PowerShell from the Start menu and run the commands below line by line. The first command checks the installed RAM, and the second command checks the graphics device name. VRAM capacity is read from Task Manager → Performance → GPU Dedicated GPU Memory. AdapterRAM values from older WMIs may not be accurate for this purpose.
Get-CimInstance Win32_ComputerSystem | Select-Object Manufacturer, Model, @{Name="RAM_GiB";Expression={[math]::Round($_.TotalPhysicalMemory / 1GB, 1)}}
Get-CimInstance Win32_VideoController | Select-Object Name, DriverVersion
Get-PSDrive -PSProvider FileSystemYou can also try running small quantized models on a PC without a GPU. This series uses 16GB RAM as an introductory starting point, but each model requires different memory. On an 8GB device, it’s easier to close other apps and start with a smaller model and shorter context.
First things to do with virtual notes
As the first input, we use a hypothetical server maintenance note. There is no need to prepare physical work documents or passwords. Save your notes and questions. You can use it to compare answers later when you change models.
서버 점검 메모: 백업은 매일 02시에 실행한다.
실패 알림은 관리자가 확인한다. 보관 기간은 14일이다.
위 메모를 넘겨짚지 말고 두 문장으로 요약해 줘.Compare each answer with the source even when it reads fluently. A summary that adds “cloud backup” when the source never mentions it has changed the facts. Factual fidelity matters more than polished phrasing. Apply the same standard to the evaluation sheet in Part 18 and the final project.
Lesson 30
Parts 1 to 7 cover installation and model selection, and parts 8 to 12 prepare Python, Node.js, WSL, and Docker. Parts 13 to 18 lead to API calls, document retrieval, and quality assessment. Hermes, OpenClaw, and OpenRouter are examined in parts 19 to 25, and the last five parts cover performance, inference engines, fine tuning, security, and operations.
Here, you just need to write down the specifications and model of your PC and one task you want to do. In the next article, run the model and use the same inputs again. When recording speed and memory, keep track of the context length and the apps that were running to make comparisons easier.
reference material
Documentation and model tags checked: October 18, 2026. Primary lab environment: Windows 11 x64.
.jpg){kind=link}