seems like a used AMD Radeon Pro V620 32 gb can be had for 400-500 range if it fits your budget/components. I was considering getting one to play with but my current setup seems to be dialed in enough where im happy enough with it.
So, how does one get more intelligence in the home lab?
SIMPLE - YOU GET MARRIED AND YOUR WIFE TAKES OVER. THE BARELY TOILET TRAINED NEANDERTHAL IS NO LONGER IN CHARGE
(Just kidding)
Well, I have a very similar setup to yours (Minisforum N5 Air instead of N5 Pro), and I chose the Radeon V620 on a Minisforum DEG01 instead of the RTX4060 mostly because it's cheap (and immediately available), has 32GB of VRAM, runs (somewhat) like an RTX3070 with decent numbers, and since it's on an Oculink dock, it can be powered off (during a reboot) independently to save me power and noise in my relatively small NYC pre-war apartment. That 41 cents/Kwh ConEd bill sure doesn't look all that appealing after the summer heat wave, and I am not running my GPU all the time unless a) I am not home or b) my wife won't kill me for the noise.
Why the N5 Air? Because it was 300 dollars cheaper than the Pro, and at the time I needed a cheap(ish) NAS enclosure to replace my supposedly dying N40L with some decent performance numbers and room to grow. While I have some underutilized DDR5 SODIMMs from before the price increases, I don't have any ECC DDR5 SODIMMs which will justify the jump to a Strix Point based AI9 HX370, although to be honest, I don't think it's needed.
I had multiple NAS crashes in the past due to the power supply dying, but none due to RAM errors, and that's with 8 years of using plain jane desktop DDR3 on my HP Microserver G7 N40L. The N40L only got a pair of 8GB ECC DDR3 DIMMs in 2021-ish after I got sick and tired of the RAM playing boot-up roulette with me, sometimes coming up with 8GB or 16GB on the RAM count - but honestly, I don't ever see parity error corrections on the system log, but I did see plenty of device brown-out kernel panics thanks to the PSU acting funny.
First, some photos!
(yeah, the
big Lego plane collection just keep growing)...
Okay, so here's what it usually looks like - note the Cobalt Qube3 Pro on top of the Minisforum - a consumer level AMD powered NAS from 20 years ago on top of a modern one. The Radeon Pro V620 is on the DEG01 dock with the 3D printed Instinct MI50 shroud + blower, and there is a BeQuiet 13M PSU powering it up. Having a Japanese F-4EJ Kai (on its Lego stand off the DEG01/BeQuiet 13M) and a Ukrainian Su-24 (on top of the Qube 3) does help set the scene of much noise and excitement.
It's a bog standard, run of the mill Minisforum NAS enclosure and it is doing yeoman duty running LLMs...
So here's what the DEG01 looked like before it became the racking solution for my planes...
Note that some cable management was done just to keep things tidy...and as my wife said, "are you sure that's not penisforum"?
How was it controlled, power-wise?
Z-Wave power plugs, one for the N5 Air and one for the DEG01.
The not-really-NAS is outfitted with a Kioxia XG8-512GB NVMe SSD and 2x64GB Crucial DDR5-5600 SODIMMs..and it uses less power than the HP t755 compute node (the N5 Air pulls 14-16W idle, about 83 when maxed - this compares to the t755 (which idles at 25 with a pair of 32GB SODIMMs, a Mellanox MCX354A ConnectX3-VPI and a single 512GB Crucial P3 NVMe SSD). The V620 is on "full send" at the moment, so to speak, eating up ~320 Watts digesting through yet another one of my less than brilliant lines of questioning when I kept asking different LLMs for cat food recipes featuring roasted garlic, or plan summer clambake recipes for the local Yeshiva, or trying to convince the LLM that Gil Hodges played Mr. Spock on Star Trek. I am polishing my LLM poisoning techniques here.
DEG01 idles at ~30-50 watts when not in use.
The concern here is that the V620 consumes more power than the rest of the rack devices combined, which can set off the overload alarm on the APC BE600M1 UPS.
So, is there a difference between when the DEG01 dock is powered up and not?
Yeah, one sounds like an office, the other sounds like an office with someone vacuuming at the end of the day...
Off ~
~ On
So, how do we switch from iGPU to V620 and vice versa? Simple - ssh in, send it a reboot command, count to 7, and then toggle the Z-Wave power plug for the eGPU dock on...or off. When the hardware boots up it'll either detect the GPU as being attached or not...and you can check accordingly usng amdgpu_top. If the V620 shows up you'll see both it and the 780M on the device list. Otherwise it'll just show the 780M.
Okay. Enough pretty pictures. What about the solid numbers?
Okay, what is in my N5 Air?
Hardware-wise?
Ryzen 7 255H (basically an AMD Zen4 HawkPoint based Ryzen 7 8845H but with its 4 TOPS XDNA1 NPU disabled), 128GB of RAM, a single 512GB SSD, DEG01 GPU dock connected via Oculink, BeQuiet PurePower 13M modular power supply,AMD Radeon Pro V620 + MI50 blower.
Software-wise?
Debian 14/Forky, , ROCm 7.2.4 with
Lemonade AI Server running the LLMs via llama.cpp and etc via backend modules (subsequently upgraded to 7.14).
How useful is XDNA1 (the neural engine on the Zen4 based Phoenix/HawkPoint APUs ) and why didn't I care about it being missing on the Ryzen 7 255H all that much?
Well, it’s not sour grapes (I have a Framework 13 with a Phoenix based Ryzen 7 7840U, so I do have access to one) - but It’s essentially a waste of silicon as things stand as of Q3 ‘26. XDNA1 is not nearly as heavily promoted or focused by devs as XDNA2 (the neural engine on the Zen5 based Strix Point/Strix Halo APUs like the one on the N5 Pro).
If AMD struggles to make XDNA2 useful to LLM, XDNA1 is that plus much weaker and even less popular. It's basically the sugar-free penis flavored lollipop of the NPU world. The Radeon i780M iGPU found on Zen4 APUs on the other hand? That gives you decent (not exciting or invigorating) performance for small LLMs, but just keep in mind that your RAM bandwidth is ~130MB/sec (around that for the N5 Pro), compared to ~250 for Strix Halo or the 200 on an Apple M1 Pro or the 300 on an M5 Pro, so while your prompt processing rate is “okay/so-so”, your sustained token traversal rates would not be great. Maybe 30-ish sustained?
Is resizable BAR turned on for better GPU throughput (keep in mind that the dock is only PCIe 4.0x4)?
Code:
root@LocalCabeza:~# lspci -v | grep -i V620
pcilib: Error reading /sys/bus/pci/devices/0000:00:08.3/label: Operation not permitted
03:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon Pro V620]
root@LocalCabeza:~# lspci -v -s:03:00 | grep -i size\=
Memory at 2800000000 (64-bit, prefetchable) [size=32G]
Memory at 27ffe00000 (64-bit, prefetchable) [size=2M]
Memory at fa000000 (32-bit, non-prefetchable) [size=512K]
Expansion ROM at fa080000 [disabled] [size=128K]
Well, that 32GB memory region will say "yes, it is". It's normally 256MB.
What are we using on the machine for inferencing?
Code:
root@LocalCabeza:~# amd-smi
+------------------------------------------------------------------------------+
| AMD-SMI 26.2.2+97f5574fe2 amdgpu version: Linuxver ROCm version: 7.2.4 |
| VBIOS version: 606253 |
| Platform: Linux Baremetal |
|-------------------------------------+----------------------------------------|
| BDF GPU-Name | Mem-Uti Temp UEC Power-Usage |
| GPU HIP-ID OAM-ID Partition-Mode | GFX-Uti Fan Mem-Usage |
|=====================================+========================================|
| 0000:03:00.0 AMD Radeon PRO V620 | 0 % 32 °C 0 10/250 W |
| 0 0 N/A N/A | 0 % N/A 16/30704 MB |
|-------------------------------------+----------------------------------------|
| 0000:c8:00.0 AMD Radeon Graphics | N/A N/A 0 N/A |
| 1 1 N/A N/A | N/A N/A 117/2048 MB |
+-------------------------------------+----------------------------------------+
+------------------------------------------------------------------------------+
| Processes: |
| GPU PID Process Name GTT_MEM VRAM_MEM MEM_USAGE CU % |
|==============================================================================|
| No running processes found |
+------------------------------------------------------------------------------+
And now, here's a bunch of numbers for your amusement and e-peen length/girth comparison, gathered via
Lemonade-eval.
Prompt processing to 512 tokens and token generation to 128 tokens are done at 5 runs each and averaged out, here's the rough testing methodology.
TESTING METHODOLOGY
Code:
time lemonade-eval -i $model load --server-url http://localhost:13305 bench -p 512 --output-tokens 128 --iterations 5 | grep real
(and then divide the number of seconds by 5, obvs - this is used for power usage testing since I'll need an ability to compare time savings persus wattage consumption to derive a rough estimate of efficiency)
There's one set for V620 on ROCm, one for V620 on Vulkan and one for the 780M iGPU for both ROCm and Vulkan. ROCm is usually slightly slower than Vulkan, but I usually stick with Vulkan for compatibility purposes. ROCm is usually fairly solid on the V620, but
on the Radeon 780M/GFX1103, it had to wait for the client apps (lemonade-ai) to progress past v11.6 and above. The situation for the Ryzen 9 AI370 and it's Radeon 890M/gfx1150 iGPU is about the same That being said, The 780M certainly punches above its weight simply because RDNA3 is a solid improvement over the RDNA2, and on a per-watt basis it delivers okay numbers. The 890M in the N5 Pro should yield 10-15% better numbers versus the 780M provided that you feed it with similar DDR5-5600 memory (the AI370 can do LPDDR5X-8500, but this is a socketed machine and the socket isn't LPCAMM2).
PERFORMANCE (FOR AMUSEMENT PURPOSES ONLY, CONSUME WITH GRAINS OF PINK HIMALAYAN SALT)
All tests are run with the context window (ctx) set to 32768, BTW.
| AI Model | V620 ROCm pp512 | V620 ROCm tg128 | | V620 Vulkan pp512 | V620 Vulkan tg128 | | Radeon 780M ROCm pp512 | Radeon 780M ROCm tg128 | | Radeon 780M Vulkan pp512 | Radeon 780M Vulkan tg128 |
| Gemma-4-E2B-it-GGUF | 3393.75 | 123.5 | | 3581.18 | 138.73 | | 879.42 | 40.48 | | 965.96 | 44 |
| Gemma-4-E4B-it-GGUF | 1869.73 | 76.4 | | 1936.78 | 82.56 | | 451.84 | 21.32 | | 490.24 | 23.6 |
| Gemma-4-12B-it-GGUF | 784.4 | 41.27 | | 762.89 | 43.5 | | 194.63 | 9.22 | | 208.85 | 10.16 |
| Gemma-4-12B-it-MTP-GGUF | 761.71 | 70.97 | | 753.3 | 69.27 | | 193.79 | 11.29 | | 209.88 | 18.85 |
| | | | | | | | | | | |
| Ministral-3-3B-Instruct-2512-GGUF | 3065.78 | 116 | | 2952.41 | 126.61 | | 777.84 | 30.03 | | 808.45 | 33.74 |
| Nemotron-3-Nano-30B-A3B-GGUF | 1764.95 | 97.32 | | 1527.42 | 105.75 | | 359.37 | 21.02 | | 379.8 | 26.5 |
| | | | | | | | | | | |
| Qwen3.6-27B-MTP-GGUF | 380.607 | 30.43 | | 395.73 | 38.92 | | 93.66 | 6.22 | | 90.26 | 9.15 |
| Qwen3.6-35B-A3B-GGUF-Q4_K_M | 1576.46 | 80.1 | | 1633.8 | 97.6 | | 327.52 | 26.47 | | 365.8 | 22.27 |
| Qwen3.6-35B-A3B-MTP-GGUF | 1656.38 | 76.48 | | 1541.15 | 103.07 | | 349.82 | 31.12 | | 360.12 | 29.95 |
| Qwen3-Coder-Next-REAM-i1-GGUF-Q3_K_M | 1213.5 | 62.5 | | 1274.67 | 79.76 | | 258.57 | 23.13 | | 273.55 | 25.24 |
| | | | | | | | | | | |
| gpt-oss-20b-mxfp4-GGUF | 1873.97 | 106.3 | | 1630.48 | 120.5 | | 383.53 | 25.36 | | 339.71 | 28.02 |
Note that the current set of numbers have the Gemma-4 35B LLMs removed due to...well, I ran out of storage and had to nuke the gguf files, and plus it wasn't too great in terms of performance.
I expected the tg128 numbers (original) to stay roughly the same as the tg512 numbers. I expect the tokenization rates to stay pretty consistent as the numbers of tokens increases and it converges to a limit. If you have a car run the Nurburgring for 5 laps versus 15 laps, if you have a good tamed racing driver with a consistent path, the lap times between 5 and 15 across multiple race day sessions will eventually converge to roughly similar numbers.
As for prompt tokenizing, there is probably something like a curve with a steep rise and a gentle slope downwards.
To visualize it, imagine pp128 and pp512 as like having your car run from 0 to 30 versus 0 to 60 and looking at its laptime.
Some cars (like the Citroen 2CV or the VW Beetle) is known for high acceleration from 0 to 30 but runs out of steam after about 35 mph, while others (family sedans) are optimized for cruising at 60, while others (like a BMW i8 or Corvette C8) can blow easily past 120. In the original pp128 tests we are only asking the hardware to evaluate the full power of the extra compute units, but in most cases the full firepower were not bought up before the prompt processing tokens were already done - by allocating extra prompt processing tokens from 128 to 512 we are actually making a more meaningful test.
The V620 is not a speed demon nVidia RTX Blackwell - in fact, I saw it being compared to an Intel Arc Pro B60 or B70 (probably somewhere between the 2), but much like the Intel Battlemage cards, it’s no slouch either - I am actually quite happy with it. It is a Radeon RX6800XT with double the VRAM (or
a Radeon Pro W6800 with no physical output) meant as cloud gaming GPUs, and certain higher-end Azure vDesktop instances use the V620 as their GPU to boost video playback. I like the idea of having an AI accelerator on an eGPU dock that I can toggle off remotely to save on power and noise. The fact that I got 32GB of GDDR6 VRAM (
360 USD acepted offer+20 for the fan/shroud + S&H) for less than
the price of 32GB of DDR5 SODIMM (419+S&H) and got a large bunch of RDNA2 compute shaders tossed in for free is...quite amusing to me. Why yes, I did have to spend 150 more on that dock, but it can be reused with another card.
What about power efficiency? Well, that will require me to compare the amount of power used versus the observed speedup in the AI compute tasks. The problem is that I don't really have an easy way to pull a total power consumed on a GPU based on a specific workload, whether it's iGPU or discrete GPU off a dock. I can look at amdgpu_top and guess from its socket power pull statistics, fudge a static wattage number (even though compute shader usage differs throughout the run)...then extrapolate it to the measured run time for a simple and naive power usage figure...
POWER/EFFICIENCY (BEST GUESSES,TAKE IT WITH FLAKES OF SEL DE GUERANDE )
All figures are given as multiples over the ones obtained over the 780M -
| Wattage, V620 ROCm versus 780M Vulkan | Speedup, V620 ROCm versus 780M Vulkan | Wattage V620 Vulkan versus 780M Vulkan | Speedup, V620m versus 780M Vulkan | Effficiency, 780M Vulkan versus V620 ROCm | Effficiency, 780M Vulkan versus V620 Vulkan |
| Gemma-4-E2B-it-GGUF | 3.469387755 | 3.086206897 | 3 | 2.701886792 | 1.28406111 | 0.972067039 |
| Gemma-4-E4B-it-GGUF | 3.469387755 | 3.483028721 | 3.265306122 | 3.237864078 | 1.071505064 | 0.937490438 |
| Gemma-4-12B-it-GGUF | 3.4 | 4.103861518 | 3.1 | 3.966537967 | 0.857170668 | 0.755386113 |
| Gemma-4-12B-it-MTP-GGUF | 3.4 | 3.735632184 | 3.2 | 3.344768439 | 1.016512821 | 0.856615385 |
| | | | | | |
| Ministral-3-3B-Instruct-2512-GGUF | 4.166666667 | 3.537414966 | 4.375 | 3.391304348 | 1.228632479 | 1.236778846 |
| Nemotron-3-Nano-30B-A3B-GGUF | 3.645833333 | 3.820895522 | 3.958333333 | 3.67816092 | 0.991210938 | 1.035970052 |
| | | | | | |
| Qwen3.6-27B-MTP-GGUF | 4.897959184 | 4.21112023 | 5 | 3.854460094 | 1.270725099 | 1.187332521 |
| Qwen3.6-35B-A3B-GGUF-Q4_K_M | 4.583333333 | 4.236509759 | 4.583333333 | 3.385321101 | 1.353884372 | 1.081865402 |
| Qwen3.6-35B-A3B-MTP-GGUF | 3.8 | 3.642172524 | 3.32 | 3.081081081 | 1.233333333 | 0.91154386 |
| Qwen3-Coder-Next-REAM-i1-GGUF-Q3_K_M | 3.06122449 | 3.374649205 | 3.06122449 | 2.738140417 | 1.117993975 | 0.907123764 |
| | | | | | |
| gpt-oss-20b-mxfp4-GGUF | 3.673469388 | 4.234113712 | 4.387755102 | 3.95625 | 0.928523068 | 1.036286553 |
| | | | | | |
| Wattage, V620 ROCm versus 780M ROCm | Speedup, V620 ROCm versus 780M ROCm | Wattage, V620 Vulkan versus 780M ROCm | Speedup, V620 Vulkan versus 780M ROCm | Effficiency, 780M ROCm versus V620 ROCm | Effficiency, 780M ROCm versus V620 Vulkan |
| Gemma-4-E2B-it-GGUF | 3.4 | 2.981132075 | 2.94 | 3.405172414 | 1.140506329 | 0.863392405 |
| Gemma-4-E4B-it-GGUF | 3.469387755 | 3.54368932 | 3.265306122 | 3.812010444 | 0.979032709 | 0.856583729 |
| Gemma-4-12B-it-GGUF | 3.4 | 4.319176319 | 3.1 | 4.468708389 | 0.787187128 | 0.693712753 |
| Gemma-4-12B-it-MTP-GGUF | 3.4 | 4.805145798 | 3.2 | 5.366666667 | 0.707574784 | 0.596273292 |
| | | | | | |
| Ministral-3-3B-Instruct-2512-GGUF | 3.921568627 | 3.753623188 | 4.117647059 | 3.915343915 | 1.044742221 | 1.051669316 |
| Nemotron-3-Nano-30B-A3B-GGUF | 3.5 | 4.431034483 | 3.8 | 4.602985075 | 0.789883268 | 0.825551232 |
| | | | | | |
| Qwen3.6-27B-MTP-GGUF | 4.8 | 5.246948357 | 4.9 | 5.73245794 | 0.914817466 | 0.854781675 |
| Qwen3.6-35B-A3B-GGUF-Q4_K_M | 4.4 | 3.041284404 | 4.4 | 3.805970149 | 1.446757164 | 1.156078431 |
| Qwen3.6-35B-A3B-MTP-GGUF | 3.8 | 3.093366093 | 3.32 | 3.656694743 | 1.228435266 | 0.907923749 |
| Qwen3-Coder-Next-REAM-i1-GGUF-Q3_K_M | 3 | 2.948766603 | 3 | 3.634237605 | 1.017374517 | 0.825482625 |
| | | | | | |
| gpt-oss-20b-mxfp4-GGUF | 3.6 | 4.115625 | 4.3 | 4.404682274 | 1.143229167 | 1.024344715 |
(Honestly, I didn't expect the numbers to change too much)
TESTING METHODOLOGY
Well, it's based on timing 5 runs of the tests above while keeping an eye on the GPU power figure on amdgpu_top. However, since different parts of the GPU (shader/assembler/runlist controller/backend interface) is used/dismissed during the process the compute figure can only be a rough guess over the span of the elapsed time. In general the power used (in watts) is multiplied by the elapsed time (in seconds), then divided by 3600 to get the approximate energy (in watt-hours) consumed. The energy consumed by a given backend of a GPU can then be compared as a ratio.
How do you read this?
The V620 consumes between 175 to 330W during the run, compared to the almost constant 47-54 W of the 780M (the V620 takes up 3.5 to 5x the power usage of the 780M, averaging about 4.2x), and the speedup is usually around 2-4.5x (averaging about 3.3x)...so we can derive the relative efficiency, and for anything over 1, it indicates that the 780M used up less power versus the V620, and if it's below 1, the V620 used less power versus the 780M. The numbers were actually closer than I would expect, but with the pattern suggesting that ROCm on the V620 being almost always less efficient than Vulkan on the 780M.
So what am I using it for? Well, mostly to mess around with local LLMs and to justify the purchase of this piece of hardware after the machine it was supposed to replace...failed to die yet again. It’s not like I plan to train a new frontier model with it.
