[Local LLM 03] Choose a Model for Your Computer

Building a Local LLM · 03/30 · Introduction

Full series and learning sequence · Practice code ZIP

Choose one task before choosing a model. Summarizing Korean notes, editing code and describing an image require different capabilities. Shortlist models that fit the computer, then compare them using questions representative of the work you intend to do.

We start with the model from Part 2 ready to run. Prepare two candidates to compare and the same questions to ask for both models.

model candidates for comparison

Use	starting candidate	Reasons for selection and limitations
General Korean/Summary	qwen3:4b-instruct	A small-scale conversational baseline model. Professional facts are verified separately
A slightly larger comparison target	qwen3:8b	Size comparison of the same series. Record differences in thinking settings
Image description	gemma3:4b	Candidate for text and image practice. Don’t confuse 1b with function
Code-only comparison	qwen2.5-coder:7b	Code-centric series. Don’t trust answers without running tests
Comparison of conversation tendencies	hermes3:8b	Comparison of dialogue and instruction implementation of Hermes 3 series
document search vector	embeddinggemma:300m	For embedding, not for answer generation.

The table above is a list of candidates for practice, allowing you to check the official distribution tags. Rather than reading it as a performance ranking of the entire model, view it as a list from which to start comparison. Even if you choose another series of Qwen or Gemma, you can still use the confirmation sequence below.

Model name and file format

Model names contain a mixture of different information. The B in 7B and 8B refers to the number of parameters in billions, and Instruct·Chat refers to modifications tailored to instructions or conversations. Base is often the starting point for further learning, so the Instruct series is convenient for first conversations. GGUF is the file format, and Q4_K_M is the notation for the quantization method.

MoE uses some of its experts when processing tokens. Even if the parameters participating in the calculation are reduced, the total weight that needs to be stored does not decrease accordingly. Be careful not to assume that you can run with the same memory as a 3B model based solely on the description that the active parameter is 3B.

Compare results with the same question

ollama pull qwen2.5-coder:7b
ollama show qwen2.5-coder:7b
ollama run qwen2.5-coder:7b

If you’re comparing code models, try actually implementing the fixes in your answer. For example, you could present a Python function that fails on an empty list, and check whether it passes the test for an empty list, a single item, or multiple items. You can see separately whether the explanation is good and whether the code has been corrected.

In the document summary, we check whether numbers that were not in the original text appear and whether proper nouns and honorifics are maintained. Initially, use the same input until the system prompt. If you change the instructions for each model, it becomes difficult to tell what caused the difference in results.

Check capacity and license before downloading

Before downloading, check the model card for the base model, supported languages, terms of use, context length and known limitations. Even for open weight models, commercial use terms may vary. When choosing a variation of Gemma·Llama, read the guidance on the variation model and the conditions of the base model together.

• Note the tags and download size, don’t just write down latest.
• We set acceptance criteria for small models and then accept larger models.
• Image, tool call, and JSON support are tested separately for each function.
• Distinguish between the size of the model file and its execution memory.

Even a small model can be enough to create a short summary on your laptop. However, agent tasks that move between multiple files and tools must be checked separately. First, compare the results of one task and expand the scope little by little.

reference material

• Qwen3 Model List
• Gemma 3 Model List
• Qwen2.5 Coder
• Hermes 3 Model Card
• EmbeddingGemma

Documentation and model tags checked: October 18, 2026. Primary lab environment: Windows 11 x64.

---