R9700 32GB eGPU + Minisforum X1 Pro-470: Qwen3.8-27B at 128k with image gen, and a counterintuitive Vulkan finding
Built this over the weekend and hit a few things that go against the usual advice, so figured I'd write it up.
\## Hardware
\- Minisforum AI X1 Pro-470 (Ryzen AI 9 HX 470, Radeon 890M iGPU)
\- 64GB DDR5-5600 (2x32, matched)
\- 2x Samsung 990 Pro 1TB
\- AMD Radeon AI PRO R9700 32GB in an AOOSTAR AG02 dock over \*OCuLink\*
\- Ubuntu Server 24.04.4, headless
\- llama.cpp + stable-diffusion.cpp, both built natively
\## The main finding: use Vulkan on the iGPU, not ROCm
This is the one I'd not seen mentioned anywhere. I wanted image generation running alongside the LLM, but a 27B model at Q4 plus Z-Image Turbo doesn't fit in 32GB together. Obvious answer: put image gen on the iGPU, which is otherwise idle.
With \*ROCm\* on the 890M: 2m09s for a 512x512 image, 8 steps.
With \*Vulkan\* on the same iGPU, same everything else: \*28 seconds\*.
4-5x faster. Vulkan reports \`uma: 1\` for the integrated GPU and appears to skip memory copies that ROCm makes on a device that shares system RAM. ROCm treats it like a discrete card.
Read lots online about "use ROCm on AMD" and that's correct for the R9700 — I use ROCm there. For integrated graphics it seems to that Vulkan is better, at least on RDNA 3.5. Maybe I missed something but for now this is working great for my useage.
Note the device-selection env vars aren't interchangeable: \`HIP_VISIBLE_DEVICES\` for ROCm builds, \`GGML_VK_VISIBLE_DEVICES\` for Vulkan. Setting the wrong one silently does nothing and you end up back on the discrete card wondering why it's fast.
Moving image generation to the iGPU freed up \~11GB on the R9700, which is what made 128k context possible.
\## Idle power: 92W -> 19W
llama-server was holding the card at full clocks doing nothing. \~90W+ constantly on an always-on box. No need for that nonsense.
Two flags fixed most of it:
\- \`--poll 0\` — stops the busy-wait. CPU went from 86% of a core to 0.6%.
\- \`--sleep-idle-seconds 60\` — unloads the model after a minute idle, releases VRAM.
That got VRAM freed but the card still sat at 3400MHz. Added a small script that polls VRAM usage and flips \`rocm-smi --setperflevel\` between \`low\` and \`auto\` depending on whether the model is resident.
Result: \*19W idle, 6.2s cold start\* on the first message of a session. Worth it for \~640 kWh/year.
Worth knowing: \`rocm-smi\` will still report GPU 100% while idle. Using amdgpu_top shows why — the command processor spins on an empty queue while every actual shader engine sits at 0%. It's a reporting artefact, power and clocks are the truth.
\## MTP speculative decoding is worth the effort
Unsloth ship an MTP module for Qwen3.8-27B in a separate \`MTP/\` folder in the GGUF repo. 1.3GB.
\--spec-type draft-mtp
\--spec-draft-model .../MTP/mtp-Qwen3.8-27B-Q4_0.gguf
\--spec-draft-n-max 3
Baseline tg128 without it: 24.8 t/s (Q5)
With MTP on real generations: \*38-52 t/s\* depending on workload - it made a huge difference to how it feels.
Draft acceptance runs 55-79%. Reasoning-heavy output accepts better than short answers — makes sense, it's more predictable. I tested n-max 2/3/4 and 3 was best for me, though the differences were a few percent.
\## Q4 vs Q6: Q4 still behaves on my workflows.
I assumed I'd want Q6. Ran both against a nasty multi-rule logic puzzle (nested conditional rules, some of which cancel others depending on question parity and primality). Same prompt, same output length:
| Q6_K_XL | Q4_K_XL |
| tg | 47.2 t/s | 50.4 t/s |
| pp | 799 t/s | 968 t/s |
| draft acceptance | 77% | 77% |
| VRAM @ 80k | 92% | 69% |
| answers | all correct | all correct |
Q4 is faster on both and uses 23 %points less VRAM. Unsloth's UD quants seem to hold up genuinely well on dense models. I'd previously seen bad hallucination from a \*\*MoE\*\* at Q4 — that's a different situation, each expert has fewer params so quantisation hits harder.
\## Final numbers
Qwen3.8-27B-UD-Q4_K_XL, 128k ctx, q8_0 KV, flash attention, MTP n-max 3, vision (mmproj), 2000 token reasoning budget:
\- pp512: 968 t/s
\- tg: 38-52 t/s in real use
\- VRAM: 75% of 32GB
\- Image gen (iGPU, Vulkan): \~28s per 512x512
\- Idle: 19W
Both models coexist. Web search, RAG, vision and image generation all work in one conversation. Access is Open WebUI behind \`tailscale serve\`, plus OpenCode on my laptop for coding work.
\## Other things that cost me time
\- \`nomodeset\` was needed to get through the Ubuntu installer\* on this hardware (console/framebuffer issue), and \*must be removed after\*, or amdgpu never loads and ROCm silently doesn't work.
\- The console goes dark during boot once amdgpu loads. as expected I guess. Use SSH.
\- ROCm needs the \*DKMS driver\*. Installing with \`--no-dkms\` leaves \`rocminfo\` reporting "ROCk module is NOT live" while everything looks superficially fine.
\- \`--parallel\` defaults to 4, which quadruples your KV cache. Set it to 1 if you're the only user. This was invisible to me for a while and I was blaming context size for VRAM pressure that wasn't context's fault.
\- \*OCuLink power limit\*: there are reports of AMD dGPUs being capped to the APU's TDP over OCuLink. Seems that's hat's a \*Windows driver\* issue but I never tested Windows so I can't confirm — on Linux mine draws the full 300W. Confirmed Gen4x4 link speed via \`amdgpu_top\`.
\- VAE decode needs a big compute buffer. \`--vae-tiling --vae-tile-size 16x16\` was the difference between working and OOM when things were tight.
Hope somebody gets some help from this.
