Install Ollama, download a small model and check a local response before adding another application. CPU execution is a valid starting point when no supported GPU is available. This guide uses qwen3:1.7b to follow terminal chat and an HTTP request to the local API.
The target is a Linux PC/server that manages services with systemd, such as Ubuntu 24.04 LTS or Rocky Linux 9. Python and Docker do not need to be installed. Downloading the model requires internet, and running the public local model in the example below does not require an Ollama account or API key. (Local API authentication guide)
On this page
Check the Linux environment before running
First, check the operating system, CPU structure, and available memory and storage space. systemd is a component that manages the startup and termination of background services in Linux. The service commands in this article cannot be applied to places where systemd does not operate, such as containers or minimal installation environments.
cat /etc/os-release
uname -m
ps -p 1 -o comm=
free -h
df -h /uname -m is usually displayed as x86_64 on general PCs and aarch64 on ARM64 devices. The official installation script receives the files by distinguishing this structure. Also check that the name of PID 1 is systemd. For other structures or special equipment, please refer to Ollama Linux installation document for support.
In free -h, you see the value available. The space to store the model and the memory to run it are separate, so even if there is ample disk space, it may not run if there is not enough memory. To use a GPU, you must also check the dedicated memory of the graphics card, VRAM, and driver support.
Install Ollama and start service
The preparation commands below will only run those appropriate for your distribution. These are the steps to prepare the download tool curl, HTTPS certificate, and zstd for decompression. Regular users require sudo permission.
Ubuntu/Debian series
sudo apt-get update
sudo apt-get install -y curl ca-certificates zstdRocky Linux/RHEL series
sudo dnf install -y curl ca-certificates zstdOnce you’re ready, run the script from ‘s official Linux installation guide, . This command directly runs a script received from the Internet and installs an executable file and service account on the system. If you have an existing installation, it may be updated.
curl -fsSL https://ollama.com/install.sh | shThe installer may attempt GPU-related installation steps depending on the hardware and environment. On an existing GPU server, review the installation script and current driver configuration first. Distribution-specific GPU driver installation is outside this guide’s scope.
Check whether the systemd service is registered and start it. enable --now starts the service now and sets it to run automatically on the next boot.
ollama --version
sudo systemctl enable --now ollama
systemctl is-active ollama
curl -fsS http://127.0.0.1:11434/api/tagsIf is-active outputs active and the last command returns JSON, the server is receiving the request. If there is no model received yet, it is normal even if the models array is empty. Model List API Checks whether the response and whether the model actually generates an answer are checked in the next step.
If the service is already running, there is no need to additionally run ollama serve in the same terminal. Conflicts may occur if you try to use the same address and port again.
Get your first answer with a small model
Start with a small model to verify the response path. The sizes below are download sizes shown on the official model pages on October 22, 2026. They are not minimum RAM or VRAM requirements, and may change when a model tag is updated.
| model tag | download size | Selection criteria |
|---|---|---|
qwen3:0.6b | About 523MB | Check the execution path first when memory is low |
qwen3:1.7b | About 1.4GB | The first conversation model we will use in this article |
qwen3:4b | About 2.5 GB | View and compare resource slack after running a small model |
A notation such as :1.7b after the model name is Tag, which specifies which variant to use. To receive a model like the example, enter tags. Finish the download with pull and open the conversation with run. (Ollama CLI command)
ollama pull qwen3:1.7b
ollama run qwen3:1.7bWhen the conversation field appears, enter a short question like the following: The sentences below are not shell commands, but questions to send to the model.
리눅스에서 RAM과 디스크 공간의 차이를 초보자에게 세 문장으로 설명해 주세요.The answer may vary each time you run it. Qwen3 is a model that can display the thought process before the answer, so there is no need to judge it as an installation error if that part is output. The first request also includes loading the model into memory.
To end the conversation, enter /bye in the input field. Normal conversations on downloaded public local models can be run without an Internet connection, but downloading new models or calling cloud models requires a network. Check whether the commands created by the model match the actual environment before execution.
Check CPU and GPU usage status
Just because an answer comes from a PC with a GPU does not mean that it is using a GPU. Immediately after talking to the model, run the following command in another terminal: ls shows the model received on disk, and ps shows the model currently loaded into memory.
ollama ls
ollama ps| PROCESSOR indication | meaning |
|---|---|
100% CPU | Load the model into system memory and run it on the CPU |
100% GPU | Load all models into GPU memory |
CPU/GPU ratio | Load model split into system memory and GPU memory |
The percentages in the table describe the loading status of the model and do not necessarily mean that GPU utilization will always be 100%. If ollama ps is empty, check again immediately after asking the question. After some time, the model may go down in memory. (Check GPU loading status)
If you plan to use an NVIDIA GPU, also check that the driver recognizes your GPU with the following command: There is no need to run it if you practice with a CPU or use a GPU from another manufacturer.
nvidia-smiIf the GPU is not found in the driver, check the GPU forwarding configuration of the driver and virtual machine first rather than changing Ollama settings first. Since the support conditions are different for each GPU such as NVIDIA and AMD, check with official hardware support list. In addition to the model file, a cache and execution buffer are required to hold the conversation, so the download size is not directly calculated as VRAM requirement.
Call the same model with local API
Once terminal chat works, send an HTTP request to the same model. The API lets other programs submit questions to Ollama and read the response. Run the following command on the Linux host where Ollama is installed; it uses the local address without an API key.
curl -fsS http://127.0.0.1:11434/api/chat \
-H 'Content-Type: application/json' \
-d '{
"model": "qwen3:1.7b",
"messages": [
{
"role": "user",
"content": "리눅스에서 RAM과 디스크 공간의 차이를 세 문장으로 설명해 주세요."
}
],
"think": false,
"stream": false,
"options": {
"num_ctx": 2048
}
}'The answer comes in at message.content in the JSON response, and checks that done is true. stream: false is a setting that receives a single completed JSON instead of a piece-by-piece response. think: false is a setting to turn off thought process output on Qwen3 in this example; other models may have different support. (Chat API, Thought Process Control)
num_ctx: 2048 specifies the context size to be handled in one request as 2,048 tokens. token is the unit in which the model processes text, and is not equal to the number of Korean characters. When inserting a long document, you may need a larger context, but this also increases the memory burden, so for the first exercise, check with short questions.
When using the server model on another PC
The default server address is 127.0.0.1:11434. If you call the same address from your laptop, it will point to the laptop itself and not the Linux server. Local APIs don’t require authentication by default, so in private practice you can connect with an SSH tunnel rather than opening the port to the outside world. (Default binding address, API authentication range)
Execute the command below on the connected PC with an SSH client. Replace linuxuser and 192.0.2.10 with your actual SSH user and server addresses. Port 11435 on the connecting PC must be empty.
ssh -N -L 127.0.0.1:11435:127.0.0.1:11434 linuxuser@192.0.2.10With the tunnel open, run the preceding API example from another terminal on the connected PC, but change the address to http://127.0.0.1:11435/api/chat. This method forwards requests within the SSH connection without changing the Ollama bindings on the server. The tunnel ends with Ctrl+C. (Local port forwarding option in OpenSSH)
Stopping/deleting models and setting services
Ending a conversation, unloading a model from memory, and clearing a disk file are different operations. Select and use only the commands you need. (CLI management command)
| work | command or input | influence |
|---|---|---|
| End current conversation | /bye | End dialog input, retain model file |
| Release model memory | ollama stop qwen3:1.7b | Unload the running model, leaving behind the model file |
| Delete model file | ollama rm qwen3:1.7b | Delete local model, download required to rewrite |
| Stop Ollama server | sudo systemctl stop ollama | API requests from other clients are also interrupted. |
The default model storage path for the official Linux service method is /usr/share/ollama/.ollama/models. The server you run directly or the environment where OLLAMA_MODELS is specified may be different. If storage space is insufficient, check the path and execution account in model storage location and offline execution guide before moving files arbitrarily.
The environment variables of Ollama run by systemd are not changed by just export in a general terminal. If you want to turn off the cloud feature to only use the local model, you can edit the service settings as follows: This setting is optional and does not prevent model downloads.
sudo systemctl edit ollamaIn the editor, preserving existing override settings, place the following: If there is already a section like [Service], add an environment variable line below it.
[Service]
Environment="OLLAMA_NO_CLOUD=1"After saving, reread the settings and restart the service. Since ongoing inference requests may be disconnected during a restart, apply when no requests are in use.
sudo systemctl daemon-reload
sudo systemctl restart ollama
systemctl is-active ollamaOfficial FAQ explains that the cloud model and web search function can be turned off with OLLAMA_NO_CLOUD=1. The Linux service environment variable is applied as systemctl edit, and after setting, check the application status in the log.
Check connection/download/memory errors
When installation is blocked, it is easy to narrow down the cause by checking server connection, model download, and inference execution separately. First, look at the service status and recent logs to determine at what stage it actually failed. (Ollama troubleshooting document)
systemctl status ollama --no-pager
sudo journalctl -u ollama -n 80 --no-pager
curl -fsS http://127.0.0.1:11434/api/tags| symptoms | What to check first | next action |
|---|---|---|
ollama: command not found | Whether the installation script is complete and command -v ollama | Check installation errors and executable file path |
connection refused | Service status, running PC and request address | Recheck local API after server startup |
address already in use | 11434 Use of existing Ollama services or other processes. | Check running servers, clean up duplicate runs |
| Model not found | Tag spelling with ollama ls | pull on the same server with the exact tag |
| Model download failed | DNS/HTTPS connection, proxy/CA certificate, storage space | After checking the failure log, retry the same pull |
| Insufficient memory/slow response | free -h, ollama ps, conversation context size | Comparison with smaller models and shorter inputs |
If you receive a message that the service is not registered, check that the installation is complete and that PID 1 is systemd. In a manual run environment without systemd, keep ollama serve in one terminal and run the model in another. Do not run servers redundantly by mixing them with the service management commands in this article.
If memory is tight, download qwen3:0.6b separately and compare. When changing, set the pull·run command and the model value of the API to the same tag. Avoid repeatedly installing drivers without verifying GPU support or turning off certificate verification to resolve download errors.
Related articles and official documentation
Once you’ve checked the first answer and the API response, select one short input for the task you want to use and compare the models. If you want to talk in a browser, you can see Open WebUI and Ollama connection, and if you want to call it from a program, you can see Calling Ollama API with Python. The full series Local LLM, structured on Windows, can also be referenced for model selection and API learning.
- Ollama Linux Installation·Service·Update
- Ollama CLI command · Chat API
- GPU Hardware Support·Troubleshooting and Logs
- Memory load/save location/service environment variables FAQ
Document/model tag confirmation date: October 22, 2026. This article is an implementation guide written after checking the official documentation and installation script. It does not include records of running the entire process from installing Ubuntu/Rocky Linux to model response on actual equipment or measuring GPU performance.
.jpg){kind=link}