Conclusions

What the project establishes, and the engineering findings behind it. No performance numbers — throughput, latency, and energy stay embargoed until the papers are published (see Results and Limitations).

What this establishes

Open-weight Polish clinical language models — the Llama-PLLuM family and Bielik — run on-premise on a consumer dual AMD Radeon AI PRO R9700 workstation (64 GB VRAM) under vLLM / ROCm, with no patient data leaving the building. The project ships three concrete artifacts: a set of public AWQ checkpoints (the first vLLM-native W4A16 quantization of each variant, to the author's knowledge), a shared SmPC calibration dataset, and a documented, reproducible method with its public engineering envelope (weight footprint, KV-cache budget, max-concurrency, Gate-1 sanity).

The single-card AWQ variants (8B, 12B) leave the second R9700 free for a parallel sweep or co-located training; the 70B family uses both cards at TP = 2. This is a deployment profile a clinic or department can actually own and run.

Engineering findings

Bringing these models up on RDNA 4 (gfx1201) surfaced three runtime facts — engineering notes only, with no performance numbers. First, enforce_eager is mandatory: with hybrid attention the default CUDA-graph path crashes on gfx1201 (HSA_STATUS_ERROR_INVALID_PACKET_FORMAT), and setting enforce_eager=True resolves it — a runtime constraint, not a model defect. Second, on this gfx1201 / ROCm 7.2.0 / vLLM 0.19.0 build the AWQ kernels run slower than BF16 for the same model; the RDNA 4 kernels are still young, and the size of that gap is embargoed. Third, the Gate-1 sanity check bypasses the chat template, sending prompts straight to /v1/completions, because the chat-template path produced failures that came from the harness rather than the model — so the documented pattern skips it.

What remains

The performance story — per-N throughput, latency distributions, energy per token, and any quantization-vs-quantization comparison — is held back until peer-reviewed publication. The method page documents the plot design; the numbers fill in by a data swap once the papers are out, with the layout unchanged.