Conclusions

What the project establishes, and the engineering findings behind it. No performance numbers. Throughput, latency, and energy stay embargoed until the papers are published (see Results and Limitations).

What this establishes

Open-weight Polish clinical language models (the Llama-PLLuM family and Bielik) run on-premise on a consumer dual AMD Radeon AI PRO R9700 workstation (64 GB VRAM) under vLLM / ROCm, with no patient data leaving the building. The project ships three concrete artifacts: a set of public AWQ checkpoints (the first vLLM-native W4A16 quantization of each variant, to the author's knowledge), a shared SmPC calibration dataset, and a documented, reproducible method with its public engineering envelope (weight footprint, KV-cache budget, max-concurrency, Gate-1 sanity).

The single-card AWQ variants (8B, 12B) leave the second R9700 free for a parallel sweep or co-located training; the 70B family uses both cards at TP = 2. This is a deployment profile a clinic or department can actually own and run.

Engineering findings

Bringing these models up on RDNA 4 (gfx1201) surfaced three runtime facts. These are engineering notes only, with no performance numbers. First, enforce_eager is mandatory: with hybrid attention the default CUDA-graph path crashes on gfx1201 (HSA_STATUS_ERROR_INVALID_PACKET_FORMAT), and setting enforce_eager=True resolves it. That is a runtime constraint, not a model defect. Second, on this gfx1201 / ROCm 7.2.0 / vLLM 0.19.0 build the AWQ kernels run slower than BF16 for the same model; the RDNA 4 kernels are still young, and the size of that gap is embargoed. Third, the Gate-1 sanity check bypasses the chat template, sending prompts straight to /v1/completions, because the chat-template path produced failures that came from the harness rather than the model, so the documented pattern skips it.

What remains

The performance story, meaning per-N throughput, latency distributions, energy per token, and any quantization-vs-quantization comparison, is held back until peer-reviewed publication. The method page documents the plot design; the numbers fill in by a data swap once the papers are out, with the layout unchanged.