How is everyone setting up Local AI in thier home labs? Idea thread.

[marcoi]

well some more tweaking on the b70 on my n5pro
the extra stuff in the set commands seem to up the performance a lot.
Im still working on the final tweaks so i can run two agents with max performance.
rest of the n5pro (npu/igpu) near max performance so unlikely ill get more out of tweaking.
I may pickup another b70 card and convert htpc with amd 5950 cpu and 128GB Ram into dedicated business AI box.
Just not sure i want the extra wattage as the n5pro combo b70 working well and low wattage.

:: 1. INITIALIZE INTEL ONEAPI PATH
call "C:\Program Files (x86)\Intel\oneAPI\setvars.bat" -arch=auto

:: 2. OVERRIDE WITH YOUR PERFORMANCE TUNING CONFIGURATIONS
set GGML_SYCL_PINNED_MEMORY=1
set ONEAPI_DEVICE_SELECTOR=level_zero:0

:: FIXES THE 16GB LIMIT: Force Intel drivers to recognize the full 32GB VRAM allocation pool
set UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1
set ZEX_NUMBER_OF_DEVICE_ALLOCATIONS_PER_POOL=0

c:\Users\marco\DEV\llama.cpp\build\bin>llama-bench.exe -m "M:\models\lmstudio-community\Qwen3.6-35B-A3B-GGUF\Qwen3.6-35B-A3B-Q4_K_M.gguf" -ngl 99 -p 4096 -n 2048 --cache-type-k q8_0 --cache-type-v q4_0 --flash-attn on -b 4096 -ub 4096 -t 2
| model                          |       size |     params | backend    | ngl | threads | n_batch | n_ubatch | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | ------: | -------: | -----: | -----: | --: | --------------: | -------------------: |
| qwen35moe 35B.A3B Q4_K - Medium |  19.70 GiB |    34.66 B | SYCL       |  99 |       2 |    4096 |     4096 |   q8_0 |   q4_0 |   1 |          pp4096 |     1555.51 ± 125.16 |
| qwen35moe 35B.A3B Q4_K - Medium |  19.70 GiB |    34.66 B | SYCL       |  99 |       2 |    4096 |     4096 |   q8_0 |   q4_0 |   1 |          tg2048 |         84.05 ± 1.40 |

build: 3dc7285b4 (10495)

c:\Users\marco\DEV\llama.cpp\build\bin>llama-bench.exe -m "M:\models\lmstudio-community\Qwen3.6-35B-A3B-GGUF\Qwen3.6-35B-A3B-Q4_K_M.gguf" -ngl 99 -p 512 -n 128 -b 512 -ub 512 --flash-attn on -t 4
| model                          |       size |     params | backend    | ngl | threads | n_batch |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | ------: | --: | --------------: | -------------------: |
| qwen35moe 35B.A3B Q4_K - Medium |  19.70 GiB |    34.66 B | SYCL       |  99 |       4 |     512 |   1 |           pp512 |      1162.31 ± 39.09 |
| qwen35moe 35B.A3B Q4_K - Medium |  19.70 GiB |    34.66 B | SYCL       |  99 |       4 |     512 |   1 |           tg128 |         97.97 ± 0.55 |

build: 3dc7285b4 (10495)

 

[marcoi]

well good news. two cards with rebar enabled on the n5pro using extra oculink card.
PS not sure why gpuz showing intel memory ddr4 lol...

 

[marcoi]

latest bench with gen38

llama-bench.exe -m M:\models\llama.cpp\Qwen3.8-27B-UD-Q4_K_M.gguf -sm layer -mg 0,1 -ts 50,50 --flash-attn on -p 512 -n 128 -t 2
| model                          |       size |     params | backend    | ngl | threads |   main_gpu |  fa | ts           |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | ---------: | --: | ------------ | --------------: | -------------------: |
| qwen35 27B Q4_K - Medium       |  15.32 GiB |    27.32 B | SYCL       |  -1 |       2 |          0 |   1 | 50.00        |           pp512 |        299.92 ± 0.45 |
| qwen35 27B Q4_K - Medium       |  15.32 GiB |    27.32 B | SYCL       |  -1 |       2 |          0 |   1 | 50.00        |           tg128 |         21.08 ± 0.02 |
| qwen35 27B Q4_K - Medium       |  15.32 GiB |    27.32 B | SYCL       |  -1 |       2 |          0 |   1 | 50.00        |           pp512 |        296.66 ± 0.11 |
| qwen35 27B Q4_K - Medium       |  15.32 GiB |    27.32 B | SYCL       |  -1 |       2 |          0 |   1 | 50.00        |           tg128 |         21.12 ± 0.05 |
| qwen35 27B Q4_K - Medium       |  15.32 GiB |    27.32 B | SYCL       |  -1 |       2 |          1 |   1 | 50.00        |           pp512 |        299.51 ± 0.40 |
| qwen35 27B Q4_K - Medium       |  15.32 GiB |    27.32 B | SYCL       |  -1 |       2 |          1 |   1 | 50.00        |           tg128 |         21.14 ± 0.07 |
| qwen35 27B Q4_K - Medium       |  15.32 GiB |    27.32 B | SYCL       |  -1 |       2 |          1 |   1 | 50.00        |           pp512 |        296.49 ± 0.19 |
| qwen35 27B Q4_K - Medium       |  15.32 GiB |    27.32 B | SYCL       |  -1 |       2 |          1 |   1 | 50.00        |           tg128 |         21.05 ± 0.04 |

build: 85c55223c (10726)

 

[marcoi]

update 9-1-26:
I think i settled into setup i want for my n5pro.

• RUN SPLIT CO-HOST ENGINE (Isolated Dual-Model Concurrent Array)
  • Focus: High-concurrency cluster layout assigning isolated targets per card.
    • B70 Card 0: Qwen 3.8 Dense (27B) ^| 1 Lane x 64k
      • Serious Dev work via hermes CLI.
    • B70 Card 1: Qwen 3.6 MoE (35B) ^| 3 Lanes x 64.2k Context Pool.
      • Quick tasks for two telebot based hermes agents.
      • Secondary tasks/backup for hermes CLI
    • AMD 890m: Gpt-oss-20b ^| 1 Lane x 131K
      • Context compression for all three agents.
    • NPU: gemma4-it-e4b-FLM and embed-gemma-300m-FLM
      • Honcho memories and title generation for hermes agents.

After tinkering for a bit, i decided best way to use my cards is setup above. I tried using both cards running MOE or 27B and they didnt really fit the needs of different agents. Either it was too slow or too stupid. Now the smart dev works on 27B model which is super at coding tasks and the telebots do quick work using the MOE which is fine for simple things like HA tasks etc.

I also set my 890m to 2GB at bios level so max of 95GB ram is available to split between the cards/models memory. Right now the box is hovering around 80 GB ram used and 5-10 % CPU.

amd 890m is doing 4-600 prompt processing and 25 t/s, but it depending on how big context gets sent. Usually takes about 30-45 seconds to process 60-80k context.

Card0 - QWEN3.8 is doing around 300-360 prompt processing and 11-20 t/s on real coding tasks.
Card1 - QWEN3.6 is doing around 600-800 prompt processing and 26 t/s. Not sure if aggregating when it running 2 or more tasks at the same time.

graffana graph.

 

[marcoi]

its been a while so i figure i provide some updates.

System been rock solid after making some adjustments. When i added the 2nd dock/b70 card i was getting a lot of windows crashing. Not sure why but i figured i try moving the deg2 dock with asrock b70 oculink to the port directly on the n5pro and put the deg1 with intel b70 on the pci oculink card. Since doing that the system has not crashed once. No idea why but if on the off chance someone else doing the same build might be worth noting.

Some cool new updates.

• Each b70 card is now running Qwen3.8-27B-UD-Q4_K_XL.gguf
  • llama command: llama-server.exe -m "%MODEL_DIR0%\Qwen3.8-27B-UD-Q4_K_XL.gguf" --mmproj "%MODEL_DIR0%\mmproj-BF16.gguf" -c 65536 -np 1 -ngl 99 --host 0.0.0.0 --port 1235 --metrics --cache-type-k q4_0 --cache-type-v q4_0 --flash-attn on -b 2048 -ub 1024 -t 4 --temp 0.7 --top-p 0.80 --top-k 20 --presence-penalty 1.5 --image-min-tokens 1024 --cors-origins "http://192.168.1.*" --jinja --chat-template-kwargs "{\"reasoning_effort\":\"none\"}" --log-file "%LOG_DIR%\card1.log"
  • So i can run two agents at the same time. I gave up on the three agents as it wasnt really working. I also gave up on qwen3.6 moe as it was having issues.
• I had hermes help me build and deploy a local self hosted firecrawl docker that running on my qnap nas. It setup with networking on opnsense with a wireguard vpn to proton vpn secure core.
  • This allows both agents to search and grab web data while protecting my local isp ip from getting blocked. Also keeps searching private.
• Im also building a pxe based llm image app that i can use to boot pcs from network into ubuntu and run llama/llm models without messing with installing anything. This is still a work in progress and its been working out kinks with secure boot and booting up the image correctly.
• For work I made a WebsphereND admin app that reads from the Dmgr info and helps to review servers, apps, settings etc. It has a report feature to document all the main details a WAS admin would need. I am also building a profile backup and restore feature.

So far well worth the investment in hardware, especially since i got the n5pro before ai price hikes.

Hopefully others are having fun playing with local AI stuff as well.

 

[WANg]

its been a while so i figure i provide some updates.

System been rock solid after making some adjustments. When i added the 2nd dock/b70 card i was getting a lot of windows crashing. Not sure why but i figured i try moving the deg2 dock with asrock b70 oculink to the port directly on the n5pro and put the deg1 with intel b70 on the pci oculink card. Since doing that the system has not crashed once. No idea why but if on the off chance someone else doing the same build might be worth noting.

Some cool new updates.

• Each b70 card is now running Qwen3.8-27B-UD-Q4_K_XL.gguf
  • llama command: llama-server.exe -m "%MODEL_DIR0%\Qwen3.8-27B-UD-Q4_K_XL.gguf" --mmproj "%MODEL_DIR0%\mmproj-BF16.gguf" -c 65536 -np 1 -ngl 99 --host 0.0.0.0 --port 1235 --metrics --cache-type-k q4_0 --cache-type-v q4_0 --flash-attn on -b 2048 -ub 1024 -t 4 --temp 0.7 --top-p 0.80 --top-k 20 --presence-penalty 1.5 --image-min-tokens 1024 --cors-origins "http://192.168.1.*" --jinja --chat-template-kwargs "{\"reasoning_effort\":\"none\"}" --log-file "%LOG_DIR%\card1.log"
  • So i can run two agents at the same time. I gave up on the three agents as it wasnt really working. I also gave up on qwen3.6 moe as it was having issues.
• I had hermes help me build and deploy a local self hosted firecrawl docker that running on my qnap nas. It setup with networking on opnsense with a wireguard vpn to proton vpn secure core.
  • This allows both agents to search and grab web data while protecting my local isp ip from getting blocked. Also keeps searching private.
• Im also building a pxe based llm image app that i can use to boot pcs from network into ubuntu and run llama/llm models without messing with installing anything. This is still a work in progress and its been working out kinks with secure boot and booting up the image correctly.
• For work I made a WebsphereND admin app that reads from the Dmgr info and helps to review servers, apps, settings etc. It has a report feature to document all the main details a WAS admin would need. I am also building a profile backup and restore feature.

So far well worth the investment in hardware, especially since i got the n5pro before ai price hikes.

Hopefully others are having fun playing with local AI stuff as well.

Yeah, I am getting roughly the same ballpark on Qwen 3.8 27B dense on my Radeon Pro V620...about 320pp/30tps - more pp on ROCm, more tps on Vulkan, but maybe about 5% variance on either directions.

To be honest, I had Lemonade-AI act as an inferencing backend to AnythingLLM just so it can be used with MCPs, and in particular, I want to see if getting a web scraper skill will improve the quality of the inferencing generated (in particular dealing with the hallucinations)...let's just say that I still think of LLMs as a nifty toy but not necessarily mission critical.

 

[Greg_E]

let's just say that I still think of LLMs as a nifty toy but not necessarily mission critical.

You should have been in the meeting I had yesterday morning... Had people with zero code experience raving about having AI write them a bunch of php to do things and creating bots that do things. Not one of them understood what the AI was really producing, but let's just run as fast as we can with the scissors. So little care about sanitizing PPI, so little care about safety and security on the network. It's very disturbing. We are going to get hacked or leak a bunch of PPI, and I'm not in a position to stop any of it other than to keep preaching caution.

 

[marcoi]

You should have been in the meeting I had yesterday morning... Had people with zero code experience raving about having AI write them a bunch of php to do things and creating bots that do things. Not one of them understood what the AI was really producing, but let's just run as fast as we can with the scissors. So little care about sanitizing PPI, so little care about safety and security on the network. It's very disturbing. We are going to get hacked or leak a bunch of PPI, and I'm not in a position to stop any of it other than to keep preaching caution.

make sure you document your efforts so they dont point fingers at you later if your in that position.
It's funny how companies jump onto the tech without knowing anything while us home lab users are working out how to run this stuff local and as private/secure as possible.

 

[marcoi]

im thinking of replacing my open webui docker with AnythingLLM. not really impressed with open webui. - might be my lack of understanding of features etc.

 

[MBastian]

... bunch of php ...

Of all languages they "chose" PHP?!
Claude and other top tier LLMs are very good at dissecting code and finding security issues IF the user has the experience and mindset to ask the right questions. People tend to listen when you present them with hard evidence.

 

[Greg_E]

Whoops, should have been python.

And one department is doing things right, they bought some kind of local AI device, I've been told it is small. I bet it's more than a factor of 10 better than what I have, probably better than Spark performance. Nobody I know has knowledge of this thing, I probably know more and not even my job to deal with it.

 

[WANg]

im thinking of replacing my open webui docker with AnythingLLM. not really impressed with open webui. - might be my lack of understanding of features etc.

Yeah, although I am not entirely pleased with AnythingLLM so far (I am starting to play with a web scraper MCP on Hermes). I had instances of having Qwen 3.8 27B run through a web scrape and an image search on anythingLLM, and still manage to not pick up on the nuance that it's still wrong. Why use a subway map to test an LLM? Because this is generic enough data, and it's also data that's easily verifiable by a human being. It should also not be something that RAGs and web scraper MCPs have issues quickly verifying.






(Oh, so how did it miss a bunch of stations after A12?)

I mean, one of my ongoing low priority projects is to build an idiot savant, so to speak...that is, pair an older, smaller tool-calling LLM (llama3-1b-instruct being my favorite but I don't think it can work with tools), along with a bunch of external skills just to make it perform better/utilize older hardware more effectively...at least something that won't require me to fire up the V620 and will be able to resist my attempts to mess with it...

 

[marcoi]

Yeah, although I am not entirely pleased with AnythingLLM so far (I am starting to play with a web scraper MCP on Hermes). I had instances of having Qwen 3.8 27B run through a web scrape and an image search on anythingLLM, and still manage to not pick up on the nuance that it's still wrong. Why use a subway map to test an LLM? Because this is generic enough data, and it's also data that's easily verifiable by a human being. It should also not be something that RAGs and web scraper MCPs have issues quickly verifying.

View attachment 50893

View attachment 50892

View attachment 50891
(Oh, so how did it miss a bunch of stations after A12?)

I mean, one of my ongoing low priority projects is to build an idiot savant, so to speak...that is, pair an older, smaller tool-calling LLM (llama3-1b-instruct being my favorite but I don't think it can work with tools), along with a bunch of external skills just to make it perform better/utilize older hardware more effectively...at least something that won't require me to fire up the V620 and will be able to resist my attempts to mess with it...

you probably mentioned it somewhere but what is the exact question you ask the llm to see if it can answer it? i want to try it out on my hermes setup with firecrawl.

 

[luckylinux]

To be honest there are many Things I am not fully clear about (both in Terms of Hardware and Software) but here is my Setup .

Hardware (typically each Host has 2-3 GPUs only, though I have a Supermicro 4028GR-TRT where I am evaluating which GPU to exactly install, since I could install up to 8-10x of them):

• AMD Radeon Pro V620 32GB
• AMD Radeon Instinct Mi50 16GB
• AMD Radeon Instinct Mi50 32GB

Ollama (initially), moved for the most Part towards llama.cpp, though I still need to setup some CI/CD Pipeline (Forgejo Runner) to centrally and automatically build llama.cpp Docker Containers targeting several Architectures. You can let oauth2-proxy handle OIDC SSO Authentication, both for Browser and for unattended (Tools) Login using OAuth Tokens.

If you build llama.cpp from Source (probably also if you don't), you can probably (I still need to test it):

• Integrate with llama-swap in order to be able to swap Model
• Install unsloth on that Image (requires Python Virtual Environment) and link to *that* llama.cpp Instance instead of the one built-in into unsloth

LiteLLM AI Gateway with OIDC SSO (using custom_auth) in order to allow ad-Hoc Hosts:

[Bug]: LiteLLM UI redirects to /ui/login after oauth2-proxy auth, bypassing Custom UI SSO handler · Issue #19856 · BerriAI/litellm

Check for existing issues I have searched the existing issues and checked that my issue is not a duplicate. What happened? When deploying LiteLLM behind oauth2-proxy (reverse proxy) and configuring...

github.com

I recently tried to set up Hermes as a Podman Container, though I am still unsure about many Things. Where to begin to be honest .

I also recently setup Paperclip to manage multiple Agents - note that there is currently a BUG during Onboarding:

Disable Onboarding - I want to use local-only Models via LiteLLM Gateway · Issue #14104 · paperclipai/paperclip

Pre-submission checklist I have searched existing open and closed issues and this is not a duplicate. I am on the latest released version of Paperclip (or can reproduce on master). I have confirmed...

github.com

Many People in HomeLab might not care about TLS/HTTPS Encryption at all, though I think it's good to follow best Practices anyways.

So one Stack is basically:

Paperclip <----> Hermes <---> LiteLLM <--> Ollama / llama.cpp

I also have obviously some other Options e.g.:
Open-WebUI <--> LiteLLM <---> Ollama / llama.cpp

And:
zed <---> LiteLLM <---> Ollama / llama.cpp

Probably also (I still haven't used opencode or pi):
opencode / pi <---> LiteLLM <---> Ollama / llama.cpp


And also some local-only Options mainly for Troubleshooting (one Open-WebUI Instance on each Host), though with llama.cpp (instead of Ollama) having the WebUI for Debugging / Playing around might not be required anymore.
Open-WebUI <--> Ollama / llama.cpp

Now I "just" need to figure out the entire Skills, Rules, Hooks, etc and how to set these up .

Suggestions are appreciated .

 

[WANg]

you probably mentioned it somewhere but what is the exact question you ask the llm to see if it can answer it? i want to try it out on my hermes setup with firecrawl.

"@Agent Can you please name all 20 Toei Asakusa line stations, starting from Oshiage and then confirm your results with a web search?"
or
"@Agent Can you please name all 20 Toei Asakusa line stations, and then confirm your results by crawling Toei Asakusa Line - Wikipedia?"
or
"@Agent can you please analyze the image at https://upload.wikimedia.org/wikipedia/commons/9/9b/都営浅草線_停車駅案内図.svg and use it to confirm your past assumptions about the Toei Asakusa line?"

 

[Greg_E]

How "well" does the Mi50 work, I've seen them at "closer to what I can afford" prices, the V620 have come up in price again, I'd like to get a 32gb card to advance another level. Vulkan or RocM. Thinking about changes to my lab.

 

[marcoi]

@WANg This is what it came back with first question

Done — and it checks out. The Toei Asakusa Line has exactly 20 stations, so your premise was right. I pulled the official route diagram from Wikipedia (each station carries its A-number badge), which gives an unambiguous count: A-20 down to A-01 = 20 stations.

Starting at Oshiage (A-20) and heading south:

1
• Station: Oshiage
• No.: A-20

2
• Station: Honjō-azumabashi
• No.: A-19

3
• Station: Asakusa
• No.: A-18

4
• Station: Kuramae
• No.: A-17

5
• Station: Asakusabashi
• No.: A-16

6
• Station: Higashi-nihombashi
• No.: A-15

7
• Station: Ningyōchō
• No.: A-14

8
• Station: Nihombashi
• No.: A-13

9
• Station: Takarachō
• No.: A-12

10
• Station: Higashi-ginza
• No.: A-11

11
• Station: Shimbashi
• No.: A-10

12
• Station: Mita
• No.: A-09

13
• Station: Sengakuji
• No.: A-08

14
• Station: Takanawadai
• No.: A-07

15
• Station: Shinagawa
• No.: A-06

16
• Station: Gotanda
• No.: A-05

17
• Station: Togoshi
• No.: A-04

18
• Station: Nakanobu
• No.: A-03

19
• Station: Magome
• No.: A-02

20
• Station: Nishi-Magome
• No.: A-01

Confirmation: the line runs Oshiage (Sumida) ↔ Nishi-Magome (Ōta). I extracted all 20 station names by matching each unique A-badge in the Wikipedia route diagram to its name — no gaps, no duplicates, and it lines up with the line's description as a Nishi-Magome↔Oshiage route. Note the south section past Sengakuji (A-08) is shared with Keikyū through-service toward Haneda/Yokohama, but those 12 stations from Oshiage to Nishi-Magome are all Asakusa Line proper.

This is what it did to get there:

 Working — 21 min — iteration 10/150, receiving stream response
I'll look this up — but one flag before I start: I'm not sure the premise is right. Let me verify both the station count and whether Oshiage is even on that line.
 Searching the web for Toei Asakusa Line list of all station...
Oshiage checks out as a Toei Asakusa Line station (A20), and the line runs Nishi-Magome  Oshiage. Now let me get the full official station list to confirm the count:
 Reading [URL]https://en.wikipedia.org/wiki/Toei_As[/URL]...
The Wikipedia extract was mostly a donation banner, but the full page text is saved locally. Let me pull the actual station table from it:
 Searching files for Oshiage|Nishi-magome|Sengakuji|Toshin...
 Searching files for ^\|.*A\d+  Searching files for A\d+ \|  Searching files for Oshiage  Reading [URL='https://en.wikipedia.org-0b3d091df2.md']en.wikipedia.org-0b3d091df2.md[/URL] L380-709
The route/station section is around line 423. Let me read it:
I have the route diagram. Rather than read through all the icon noise, let me extract the station names and their A-numbers programmatically:
 Running code import re path = "/home/user/.hermes/...

 Searching the web for "Toei Asakusa Line" "20 stations" Nis..

 

[luckylinux]

How "well" does the Mi50 work, I've seen them at "closer to what I can afford" prices, the V620 have come up in price again, I'd like to get a 32gb card to advance another level. Vulkan or RocM. Thinking about changes to my lab.

Well Pick your Poison is how I would answer .

Mi50 Support is quite good I would say. I tried to build my own Docker Image and gave up after a Day of trying, if you just use Mixa's Image, it works pretty much out of the Box with llama.cpp and Power Limiting worked out of the Box for me on the 16GB Cards:

GitHub - mixa3607/ML-gfx906: ML software (llama.cpp, ComfyUI, vLLM) builds for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60

ML software (llama.cpp, ComfyUI, vLLM) builds for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60 - mixa3607/ML-gfx906

github.com

The 32GB though is not worth at these Prices IMHO.
I got a bunch of 16GB Mi50 and I will need to sell them to make Room for some other GPUs I ordered.

The V620 lacks built-in Power Limiting Support, though I forked a Repository that somebody mentioned on this Forum and Ported it to Proxmox VE with the Help of Claude / ChatGPT:

GitHub - luckylinux/v620_toolbox at proxmox_ve_initial_support

Melting pot of v620/gfx1030 tools and setups. Contribute to luckylinux/v620_toolbox development by creating an account on GitHub.

github.com

Heck I planned on building like 8 or so AMD AM4 Rigs with 2 GPUs (basically a cheap platform with a pair of decent PCIe 4.0 x 8 Slots and pretty much a Free Upgrade giving DDR4 Prices) each plus the big server with 8-10 GPUs, plus some 2011-3 Servers with 2-3 GPUs and more System RAM.

I wish Cases were larger, PCIe Adapters/Risers not so much a PITA and MMIO/SFF-8654 Cables not so expensive, but it is what it is .

 

[WANg]

@WANg This is what it came back with first question

Done — and it checks out. The Toei Asakusa Line has exactly 20 stations, so your premise was right. I pulled the official route diagram from Wikipedia (each station carries its A-number badge), which gives an unambiguous count: A-20 down to A-01 = 20 stations.

Starting at Oshiage (A-20) and heading south:

1
• Station: Oshiage
• No.: A-20

2
• Station: Honjō-azumabashi
• No.: A-19

3
• Station: Asakusa
• No.: A-18

4
• Station: Kuramae
• No.: A-17

5
• Station: Asakusabashi
• No.: A-16

6
• Station: Higashi-nihombashi
• No.: A-15

7
• Station: Ningyōchō
• No.: A-14

8
• Station: Nihombashi
• No.: A-13

9
• Station: Takarachō
• No.: A-12

10
• Station: Higashi-ginza
• No.: A-11

11
• Station: Shimbashi
• No.: A-10

12
• Station: Mita
• No.: A-09

13
• Station: Sengakuji
• No.: A-08

14
• Station: Takanawadai
• No.: A-07

15
• Station: Shinagawa
• No.: A-06

16
• Station: Gotanda
• No.: A-05

17
• Station: Togoshi
• No.: A-04

18
• Station: Nakanobu
• No.: A-03

19
• Station: Magome
• No.: A-02

20
• Station: Nishi-Magome
• No.: A-01

Confirmation: the line runs Oshiage (Sumida) ↔ Nishi-Magome (Ōta). I extracted all 20 station names by matching each unique A-badge in the Wikipedia route diagram to its name — no gaps, no duplicates, and it lines up with the line's description as a Nishi-Magome↔Oshiage route. Note the south section past Sengakuji (A-08) is shared with Keikyū through-service toward Haneda/Yokohama, but those 12 stations from Oshiage to Nishi-Magome are all Asakusa Line proper.

This is what it did to get there:

Working — 21 min — iteration 10/150, receiving stream response
I'll look this up — but one flag before I start: I'm not sure the premise is right. Let me verify both the station count and whether Oshiage is even on that line.
Searching the web for Toei Asakusa Line list of all station...
Oshiage checks out as a Toei Asakusa Line station (A20), and the line runs Nishi-Magome  Oshiage. Now let me get the full official station list to confirm the count:
Reading [URL]https://en.wikipedia.org/wiki/Toei_As[/URL]...
The Wikipedia extract was mostly a donation banner, but the full page text is saved locally. Let me pull the actual station table from it:
Searching files for Oshiage|Nishi-magome|Sengakuji|Toshin...
Searching files for ^\|.*A\d+  Searching files for A\d+ \|  Searching files for Oshiage  Reading [URL='https://en.wikipedia.org-0b3d091df2.md']en.wikipedia.org-0b3d091df2.md[/URL] L380-709
The route/station section is around line 423. Let me read it:
I have the route diagram. Rather than read through all the icon noise, let me extract the station names and their A-numbers programmatically:
Running code import re path = "/home/user/.hermes/...

Searching the web for "Toei Asakusa Line" "20 stations" Nis..

Close, but off by a bit.
From A10 (Nihombashi) to A20 (Oshiage) it was spot-on.
From A01 (Nishi-Magome) to A05 (Gotanda) it was correct.
A06 is Takanawadai (Not Shinagawa, which is the Keikyu station south of Sengakuji), A07 is Sengakuji, A08 is Mita, A09 is Daimon (missing).
Eh, yeah, even with a web scraper and a relatively well regarded open weight LLM, it somehow still manages to be incorrect.
So, more mitigations needed?

 

[WANg]

How "well" does the Mi50 work, I've seen them at "closer to what I can afford" prices, the V620 have come up in price again, I'd like to get a 32gb card to advance another level. Vulkan or RocM. Thinking about changes to my lab.

It's fine-ish, but the issue is that it's fundamentally a "big Vega" (i.e. kinda like a gimped nVidia Pascal). A smallish Vega is decent as an iGPU, but they are not great as discrete cards. They have the memory bandwidth, but not the compute to sustain them. There's also an assumption AMD might drop anything older than RDNA2 in the future ROCm releases. Yeah, if you gun for a 32GB card, either lowball someone for a V620, get an MI100, or buy an Intel B70 (which is similar/15% faster to a V620 but more efficient, quieter and will be supported longer with actual GPU virtual functions built-in).
To be honest, it’s really difficult to be happy about buying anything for inferencing these days.