How is everyone setting up Local AI in thier home labs? Idea thread.

[marcoi]

Close, but off by a bit.
From A10 (Nihombashi) to A20 (Oshiage) it was spot-on.
From A01 (Nishi-Magome) to A05 (Gotanda) it was correct.
A06 is Takanawadai (Not Shinagawa, which is the Keikyu station south of Sengakuji), A07 is Sengakuji, A08 is Mita, A09 is Daimon (missing).
Eh, yeah, even with a web scraper and a relatively well regarded open weight LLM, it somehow still manages to be incorrect.
So, more mitigations needed?

so i gave it the same question, a link to the image you had and a screen shot of the image since its orignal form was svg. i think it got it right?

All sources now agree, including the official Tokyo Metropolitan Bureau of Transportation site (kotsu.metro.tokyo.jp), which lists A01–A20 with A05 = Gotanda (五反田) — that resolves my one lingering doubt (I'd initially expected 19 stops without Gotanda, but the official site, Wikipedia's route table, and your SVG all confirm 20).

Toei Asakusa Line — all 20 stations, Oshiage → Nishi-magome:

1. Oshiage 押上 (A20)
2. Honjo-azumabashi 本所吾妻橋 (A19)
3. Asakusa 浅草 (A18)
4. Kuramae 蔵前 (A17)
5. Asakusabashi 浅草橋 (A16)
6. Higashi-nihombashi 東日本橋 (A15)
7. Ningyocho 人形町 (A14)
8. Nihombashi 日本橋 (A13)
9. Takaracho 宝町 (A12)
10. Higashi-ginza 東銀座 (A11)
11. Shimbashi 新橋 (A10)
12. Daimon 大門 (A09)
13. Mita 三田 (A08)
14. Sengakuji 泉岳寺 (A07)
15. Takanawadai 高輪台 (A06)
16. Gotanda 五反田 (A05)
17. Togoshi 戸越 (A04)
18. Nakanobu 中延 (A03)
19. Magome 馬込 (A02)
20. Nishi-magome 西馬込 (A01)

How I verified:
- Web search: The official Toei (kotsu.metro.tokyo.jp) station index lists A01 Nishi-magome through A20 Oshiage in exactly this order, and EkiView independently confirms all 20 with matching codes.
- Your SVG: I downloaded it, rendered it at 3× resolution (the text is vector paths, so it needed rasterizing), and read it in zoomed segments. The diagram shows A01–A20 left-to-right with the same names — including A05 Gotanda 五反田 with its JR Yamanote + Tokyu Ikegami transfer badges, Keikyu through-service at Sengakuji, and Keisei through-service at Oshiage. Your screenshot matches my render exactly.

All three sources (official Toei site, Wikipedia, and the diagram) are in full agreement: 20 stations, this order.

 

[Samii]

Great thread — lots of big iron here. Coming at this from the AI dev/automation side rather than the hardware side:

The box that's on 24/7 runs Proxmox, and the AI stack is all Docker on an Ubuntu VM with the GPU passed through:
- Ollama for local models (chat + smaller embedding models for RAG experiments)
- Open WebUI as the front end — the family-acceptance factor of a ChatGPT-like UI vs a terminal can't be overstated
- LiteLLM as an OpenAI-compatible gateway in front of everything, so scripts and apps talk to one endpoint and I can swap between a local model and a cloud API without changing code

Two practical notes from experience:
1. Don't expose Ollama/WebUI ports directly — Tailscale on the VM gives secure remote access with zero port forwarding.
2. Idle power is the real cost driver (echoing what several of you found with the 5090/3090 systems). I cap the GPU with nvidia-smi -pl for chat duty and only uncap for batch jobs — a 3090 at a ~200W cap still does ~100 tok/s on 7B models, plenty for chat.

For anyone starting out: a single 24GB card (3090/A5000) running Ollama + Open WebUI in Docker is the sweet spot. Everything else in this thread is scaling up from there.

 

[unwind-protect]

Still milling away with a single 5090 in an AMD 5950x/128GB system.

I will say that variants of Qwen that fit this are surprisingly capable. I just can't run the context window I want.

 

[luckylinux]

1. Don't expose Ollama/WebUI ports directly — Tailscale on the VM gives secure remote access with zero port forwarding.

You should anyways be putting a Reverse Proxy in front as a bare minimum (TLS).

Though yes, I only allow access to my Homelab via Wireguard VPN. Probably I'll be migrating to NetBird VPN though (self-hosted).

Still milling away with a single 5090 in an AMD 5950x/128GB system.

I will say that variants of Qwen that fit this are surprisingly capable. I just can't run the context window I want.

I'm pretty sure it's a highly capable Card. Way outside my Budget though . That and I'm not a big fan of that 12V Connector on the new NVIDIA GPUs.

 

[Greg_E]

Only downside I've had with net bird (free hosted plan) was uploads. Max speed I can get is about 4mpbs with a good 40mpbs up from my isp. The other side has 2gbps down and in the middle of the night is certainly not the issue. Don't have the ability to self host right now.