[Python Environments 05] Plan pandas, Jupyter, CPU and GPU Environments

A CSV processing job and a GPU inference server can both use Python while requiring different packages, resources and deployment schedules. Identify why an environment is growing before combining the workloads. This part separates batch processing, notebooks, CPU inference and GPU work by those requirements.

Python development environment · 01 selection and venv · 02 uv · 03 conda · 04 web Development·05 Data·AI·06 OCR·Service Separation

On this page

Choose first depending on the type of task

work	starting point	Take additional care
Small CSV·API collection batch	uv or venv. Simply focusing on Python packages	Input location, output schema, encoding, schedule
Combination of scientific calculation and native package	Compare conda as well. Consider the team’s existing analysis environment and channel criteria	Binary library, channel, platform
interactive analytics	Add notebook tool/kernel to the above environment	Kernel path, execution order, data version
GPU learning/inference	GPU-only project or environment	Driver·Framework build·GPU memory·Model revision
Combining web API and model inference	Initially, only one service is available. Separate when requirements diverge	Request Contract/Queue/Model Worker/Fault Boundary

pandas has both pip and conda installation paths. Rather than locking down tools like “I work with data, so conda”, check out the native dependencies and your team’s reproducibility process. pandas official installation document also guides you through both methods.

1. Move code, data, and laptops to different locations

data-demo/
├── pyproject.toml
├── uv.lock
├── .python-version
├── src/data_demo/
│   ├── __init__.py
│   └── summary.py
├── tests/test_summary.py
├── notebooks/
├── data/
│   ├── raw/
│   └── processed/
└── .gitignore

The original data is divided by raw, and the derived result is divided by processed. If you do not overwrite the original, it is easy to change the conversion code and process it again. Actual operational data, personal information, and large files can be managed with established storage and access policies instead of code storage, and only small test data can be left as a separate fixture.

2. Create a CSV batch with minimal package

[project]
name = "data-demo"
version = "0.1.0"
requires-python = ">=3.12,<3.13"
dependencies = ["pandas>=2.2,<4"]

[dependency-groups]
dev = ["pytest>=8,<10", "ruff>=0.11,<1"]
notebook = ["ipykernel>=6,<8"]

[build-system]
requires = ["hatchling>=1.27,<2"]
build-backend = "hatchling.build"

Make src/data_demo/__init__.py an empty file, and save the following contents in summary.py. The amount in this example is an integer in Korean Won and does not refer to a general accounting model that handles decimal amounts.

import argparse
from pathlib import Path

import pandas as pd


def summarize(source: Path, destination: Path) -> None:
    frame = pd.read_csv(source, dtype={"category": "string", "amount": "int64"})
    result = frame.groupby("category", as_index=False)["amount"].sum()
    destination.parent.mkdir(parents=True, exist_ok=True)
    result.sort_values("category").to_csv(destination, index=False)


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("source", type=Path)
    parser.add_argument("destination", type=Path)
    args = parser.parse_args()
    summarize(args.source, args.destination)

data/raw/sales.csv

category,amount
A,10000
B,7000
A,5000

uv python pin 3.12
uv lock
uv sync --locked
uv run --locked python -m data_demo.summary data/raw/sales.csv data/processed/totals.csv
cat data/processed/totals.csv

The result is A 15000, B 7000. Since the input and output paths are received as command arguments, the same code can be executed on the development PC and batch server by simply changing the file location. It’s easier to deploy than writing the absolute path to your personal PC in the code.

tests/test_summary.py

from data_demo.summary import summarize


def test_totals(tmp_path):
    source = tmp_path / "sales.csv"
    output = tmp_path / "totals.csv"
    source.write_text("category,amount\nA,10000\nB,7000\nA,5000\n", encoding="utf-8")
    summarize(source, output)
    assert output.read_text(encoding="utf-8").splitlines() == [
        "category,amount", "A,15000", "B,7000"
    ]

uv run --locked python -m pytest
uv run --locked ruff check src tests

3. The laptop selects the kernel of the same environment

uv run --locked --group notebook python -m ipykernel install --user \
  --name data-demo --display-name "Python (data-demo)"

Even if you run the Jupyter server with a different tool, the kernel can select this environment. Check sys.executable in the first cell of the notebook, and move the processing logic to be reused to src/data_demo for testing. If you are left with a state that depends solely on cell execution order, the results may vary when you “re-run from the beginning.”

If conda is the team standard, install and run the same processing code in the environment from Part 3. When conda is the project’s primary manager, do not let uv independently resynchronize that same environment. Define one reproducible procedure stating which tool owns each dependency.

4. In the AI environment, distinguish between drivers and Python packages.

floor	Example to check	Is it outside the lock file?
GPU driver	Check drivers and devices with nvidia-smi	OS/host management
Build the framework	PyTorch version, CPU/CUDA/ROCm build	Project specification and selected package sources
CUDA development tools	Check compatibility with nvcc when building custom extensions	Additional tool installation/build environment
model file	Model ID, revision·checksum, license and storage location	Separate model artifact management
Inference settings	Precision, placement, context, memory limitations	Run settings and verification records

The CUDA display in nvidia-smi indicates the maximum CUDA version supported by the driver, not a list of versions of the installed Toolkit. You must separately check the driver, runtime included in the framework, and Toolkit required for source building. Do not assume that venv or conda will install the GPU drivers for you. Please refer to NVIDIA Compatibility Description.

Configuration example for CPU-only PyTorch project

The following CPU-only specification belongs in a separate directory from the earlier pandas project. For a GPU environment, choose package sources that match the device, OS and supported Python version using uv’s PyTorch configuration guide and the PyTorch installation selector.

[project]
name = "ai-cpu-demo"
version = "0.1.0"
requires-python = ">=3.12,<3.13"
dependencies = ["torch>=2.8,<3"]

[tool.uv.sources]
torch = { index = "pytorch-cpu" }

[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true

# 위 명세를 저장한 별도의 ai-cpu-config 디렉터리에서
uv python pin 3.12
uv lock
uv sync --locked
uv run --locked python -c 'import torch; print(torch.__version__); print(torch.version.cuda); print(torch.cuda.is_available())'

explicit = true and tool.uv.sources are configured to retrieve torch from the specified CPU index and avoid the situation where even general packages are searched at that index. When choosing an index for GPU, don’t assume that it will work just by changing the CUDA number in the URL. Combination of driver and framework support comes first.

The execution verification in this series targeted CSV batch and CPU-based general Python/OCR environments. The PyTorch specification is a configuration example based on official documents and does not present actual results of GPU learning/inference performance or CUDA device operation.

When to maintain the same environment and when to share

Small experiments that call the model once can be started with one environment. However, if the API and training code require a different version of Torch, or if only the GPU worker has large model/driver requirements, an independent environment is easier to operate. A development tool group is a device that divides secondary dependencies of the same project, and is not a device that completely separates conflicting execution environments.

This standard also applies to OCR services. In the following 06 OCR/Service Separation, executable files and language data outside the Python library are also checked.

Download series example file·venv·conda specification, uv·web·data·OCR example source, and confirmed uv lock file are bundled. It is executed from each project directory, and the OS package is prepared separately through the procedures in the main text.