Building a Local LLM · 25/30 · Connect with an Agent
Full series and learning sequence · Practice code ZIP
An application can use a local model by default and send selected questions to a cloud model. Make the destination visible to the user and keep that choice explicit. This example does not silently route a failed local request to the cloud.
We use the shared files for Parts 13 and 24. External calls are only permitted for non-sensitive questions of your choosing.
Local failure and external delivery policy
Even if a local request fails due to insufficient memory, external transmission of data is not permitted. Fallback and data transfer policies for availability must be set separately. The example below simply reports a local failure and does not automatically switch.
| input conditions | path |
|---|---|
| default execution | Local Ollama |
| –Specify only cloud | Interrupted due to lack of external transmission confirmation |
| Specify –cloud –public | External model of your choice |
| local call error | Return an error and allow the user to check the cause |
The public option indicates that the user has confirmed it. We do not guarantee that sensitive information can be detected or that the data can be made public. Because data may be confidential even without a social security number, organizations separately define data classification and allowable paths.
Branch of your choice
import argparse
from llm_http import post, MODEL
from openrouter_demo import cloud_chat
parser = argparse.ArgumentParser()
parser.add_argument("--cloud", action="store_true")
parser.add_argument("--public", action="store_true",
help="외부 전송 가능한 비민감 질문임을 사용자가 확인")
args = parser.parse_args()
if args.cloud and not args.public:
parser.error("Cloud use requires explicit --public confirmation")
question = input("질문: ").strip()
if not 1 <= len(question) <= 1000:
raise SystemExit("Use a question of 1 to 1000 characters")
if args.cloud:
model = input("검토한 OpenRouter 모델 ID: ").strip()
answer, usage = cloud_chat(question, model)
print(answer)
print("route=cloud", usage)
else:
r = post("/api/chat", {
"model": MODEL, "messages": [{"role": "user", "content": question}],
"stream": False,
"options": {"num_ctx": 4096, "num_predict": 256, "temperature": 0.2}
})
print(r["message"]["content"])
print("route=local").\.venv\Scripts\python.exe .\hybrid_chat.py
# 가상 질문을 외부에 보낼 때만 실행
.\.venv\Scripts\python.exe .\hybrid_chat.py --cloud --publicRouting rules and data classification
You could have the model classify the difficulty of a request, but even that call takes time and costs money. If you leave sensitivity judgment to an external model, the data has already been sent before classification. First, we divide the paths with user choices and clear business rules.
If automatic routing is required, it is determined based on document type and length, allowed providers, and evaluation results for each model. For example, you might want to keep internal regulatory searches local, with only stylistic corrections to public documents as an external candidate. We need rules that explain what material should be sent where.
Route/token/cost record
The log records the path, model ID, processing time, status, and number of tokens before the question text. Add the cost by multiplying the input and output tokens by the unit price, and check the separate tool fee. Actual charges are matched against per-key limits and account usage.
The sample CLI does not include multiple user authentication, limits on concurrent requests, or atomic deduction of the budget. Shared services require additional processing like this. It is configured so that the user can check which of the request body and RAG basis is being sent externally.
reference material
Documentation and model tags checked: October 18, 2026. Primary lab environment: Windows 11 x64.
