Harness
leChatonFat4Navi: a local coding agent on the same two R9700 cards that carry the NaviMed benchmarks. No external API, no cloud: model, web search and tools all run on the workstation.
github.com/kicrazom/leChatonFat4Navi is a fork of leChatonFat by Mateusz Choiński (MIT licence). The fork adds AMD support. It is a companion tool: a practical demonstration of Navi 4 in use.
What it runs
- llama.cpp server with a GGUF model, OpenAI-compatible API on port 8001. Default model: Qwen3.6-35B-A3B-UD-Q4_K_XL.
- SearXNG, a self-hosted metasearch engine, for web search without a third-party API.
- MCP servers (Model Context Protocol, the standard way to plug tools into an agent): page fetch, SearXNG search, browser automation.
- A generated
opencode.jsonthat connects the Opencode coding agent to all of the above.
What the fork changes
GPU_BACKEND=auto|nvidia|rocm|vulkan|sglang|vllmpicks the engine. The first four use llama.cpp images (server-cuda,server-rocm,server-vulkan) with GGUF models.sglangruns an SGLang build for RDNA 4 andvllmrunsvllm servefrom a host virtualenv, both with Hugging Face safetensors.- Only the render nodes listed in
AMD_GPU0andAMD_GPU1reach the container, so the integrated GPU stays out of the layer split. make check_gpuprints the PCI ID of every render node, to tell discrete cards from the iGPU.MODELS_DIRpoints at an existing GGUF directory; the default Hugging Face repository name is corrected.- Unsets
TEMP(the sampling temperature) before start: with it set,llama-serveron ROCm crashes while loading the model.
Measured on 2x Radeon AI PRO R9700
Single stream, one request at a time. Qwen3.6-35B-A3B-UD-Q4_K_XL, context 98,304 tokens, KV cache q4_0, flash attention, layer split 50/50. Prompt of 600 tokens, 256 generated tokens, prompt cache off, three runs, 2026-11-13.
| Backend | Image size | Prompt processing | Generation |
|---|---|---|---|
| ROCm 7.2.1 | 30.8 GB | about 2,650 tok/s | 76.9 tok/s |
| Vulkan (RADV) | 1.2 GB | about 1,720 tok/s | 88.4 tok/s |
gpt-oss-20b (ggml-org/gpt-oss-20b-GGUF, native MXFP4, 12.1 GB), same settings, the chat template stored in the GGUF, generation forced to 256 tokens:
| Backend | Image size | Prompt processing | Generation |
|---|---|---|---|
| ROCm 7.2.1 | 30.8 GB | about 4,300 tok/s | 124.8 tok/s |
| Vulkan (RADV) | 1.2 GB | about 2,580 tok/s | 130.6 tok/s |
In April 2026 this model did not start on these cards under vLLM 0.19 (a triton_kernels API mismatch, documented in the session log). llama.cpp does not use that code path.
Qwen3.8-27B (unsloth/Qwen3.8-27B-GGUF, UD-Q4_K_XL, 17.7 GB), a dense model, same settings, 2026-11-14. Tool calls (one city and two parallel cities) worked on both backends:
| Backend | Image size | Prompt processing | Generation |
|---|---|---|---|
| ROCm 7.2.1 | 30.8 GB | about 960 tok/s | 27.8 tok/s |
| Vulkan (RADV) | 1.2 GB | about 575 tok/s | 26.6 tok/s |
On all three models ROCm reads long prompts faster. Vulkan generates faster on the two MoE models and is level on the dense one, and its image is 25 times smaller. The dense 27B generates about three times slower than the 35B MoE, which activates only 3B parameters per token. A coding agent re-reads a large context on every turn, so prompt processing usually decides.
Single machine, single model, N=3 per backend. Not a controlled benchmark in the sense of the Methodology page: no thermal gating, no power logging, two vLLM embedding services shared the first card during the runs.
Three engines, one Polish agent model
Bielik-11B-v3.1-Instruct (SpeakLeash), BF16 weights (22 GB) on both cards. Same prompt and generation length as above, three runs. Tool calling was tested through the OpenAI-compatible API: one call, and two parallel calls in a single answer. All four rows passed both tests.
| Engine | Split | Prompt (600 tokens) | Generation |
|---|---|---|---|
| llama.cpp, ROCm | layers | 211 ms | 23.7 tok/s |
| llama.cpp, Vulkan | layers | 470 ms | 24.9 tok/s |
| SGLang 0.5.18 | tensor (TP=2) | about 150 ms | 33.5 tok/s |
| vLLM 0.19 | tensor (TP=2) | 77 ms, warm cache | 22.4 tok/s |
llama.cpp splits the model by layers, so each token passes through one card and then the other. SGLang and vLLM split every layer across both cards, which is why SGLang generates fastest here. vLLM runs with the gfx1201 environment floor from the Reproduce page, which serialises kernels and slows decoding. The rows are therefore a working setup, not an engine comparison; that comparison is planned under one protocol.
Two setup notes. Tool calls need the built-in hermes parser on SGLang and vLLM (SpeakLeash's own parser is a plugin for vLLM 0.24 or newer). llama.cpp rejects the model's chat template because of a \u200b escape; the same template with the literal character in its place works.
Quick start on AMD
git clone https://github.com/kicrazom/leChatonFat4Navi
cd leChatonFat4Navi
make copy_env # then edit .env:
# GPU_BACKEND=rocm (or vulkan)
# AMD_GPU0=/dev/dri/renderD128
# AMD_GPU1=/dev/dri/renderD129
# TENSOR_SPLIT=50,50
# MODELS_DIR=/path/to/gguf
make check_gpu # which renderD is which card
make up
make smoke_test
Your user must be in the video and render groups.