Slowdown in the homelab segment

[WANg]

View attachment 48044

Oh god, UCS did an entire article about why microreactors aren't better, and how some of the "disruptive" stuff isn't really new innovations that was hyped up to be - I could tell you how poorly both Fermi 1 and Fort St. Vrain did in the past, and how most of the fundamental design issues from those plants in the past were still not solved today (yeah, I'll love to have a pebble bed reactor if the pebbles don't erode into radioactive dust, and I'll love to see more nuclear if the US is more like France where nuclear material recycling is mandatory and transauranic waste has somewhere to go. Piling up spent fuel rods at the back of old nuclear sites just seem like a VERY BAD IDEA.

I am pretty sure the drawing is from BWRX-300, a scaled down ESBWR from GE-Hitachi, and it's going to be built in Darlington, Ontario (Canada) first. It's not nearly as small as you might think, it's about as large as an early nuclear reactor like Oyster Creek NPP in New Jersey. I am not sure how making it output less energy will necessarily make it better or cheaper. Safer maybe, but more economical? Only if you do it at scale like France, and even then Areva hit a brick wall with EBR.

It's kinda scary how much you end up remembering about nuclear energy when you worked for a firm power trading NYISO and PJM...you end up reading more than you care to know about various power generation schemes.

 

[kapone]

It's money. It's always money.

Hardware prices are up, power prices are up, every moronic piece of software thinks it should be a subscription, Hell, Tesla FSD is a subscription now... Honorable mention: BMW subscription for heated seats...

If you've read/seen some of my posts from last year, I expanded my home "setup" (it's not just a lab, I run production from there as well) quite significantly. Essentially two racks of stuff.

Everything turned off except UPS/Switches/Router = Base load is ~250w
Lab running = ~900w
Production running = ~3kw
Both running = ~4kw.



But..this supports my business, so it is what it is. Once we move to Colorado, the power costs should be less than half of what it is right now, CO has cheap electric.

So, if money is the problem...the solution is also money. Just gotta make more, which I am. I cosplay as an active trader as well... The markets were already stalling out, and then we attacked Iran. The last couple of weeks have been profitable enough that I could retire, if need be.

Sorry, I'm rambling a bit.

 

[mattventura]

My main reason is the increased power consumption of newer hardware.

I dont need tripple the perf for double the consumption, I need more PCIe and memory on same/lower consumption (for faster low qd storage)
That coupled with ever increasing board/cpu prices (even before memory based inflation) made me stop buying new stuff.

O/c power costs already made me stop deploying newly bought builds before that, I didnt mind 1kw/h at €0.18 but at €0.28 ?

Unfortunately the industry is going the wrong direction for my use case (or probably a lot of home-labbers), but hopefully lower tier hw (small to med business vs enterprise) will move to a spot where its interesting again, since I don't I'd be willing to run 450W CPU's at home

Agreed. For example, my old E5-1650v2 when overclocked would pull up to about 160w. My Threadripper 7960X usually *idles* on the desktop at about 150w (it can't realistically get much C6 time due to background apps and such, but even when it does it's still ~100w). Sure, it's way more efficient under load. I've heard that the Xeon 6000-P series has good idle power, but I don't have any spare DDR5 and I'm certainly not buying any until prices recede a bit. My power bill is also up despite overall usage being down due to rate increases.

Fortunately, I have more than enough storage for the time being.

What I'm hoping happens with stuff like the Epyc 4000 series gives us the best of both worlds - smaller, idle-efficient CPUs but with enterprise features like ECC, NVMe hotplug, and so on. Part of the problem is that the consumer sockets have been stuck at the same general amount of PCIe lanes for ages (x16 or x8x8 GPU, a couple x4s for NVMe, everything else from the chipset), while enterprise stuff has moved way up - 88 PCIe lanes on Xeon 6000-P. Even Xeon D moved way upmarket, with the lowest-end SKU having a 110W TDP.

 

[marcoi]

IDK if hardware companies even care about low end consumers anymore.
Market for making money seems to be going to data center and AI. So we the end users are getting shafted with what's available or built for end users.
Software companies are moving to a subscription model for everything and soon OS will just be a small agent that connect something like mini pc with min specs up to a virtual desktop and services.

I would buy from a company that builds purpose built hardware for home lab or small business users.
I want something with 16+ power efficient cores/ Max Memory with ECC / IPMI / and more then 16 pcie lanes.
Maybe a company like minisforums might be able to build something like that if someone backs them?

 

[Greg_E]

I would agree on the homelab specs. for the most part, 8c16t is a pretty good punch, and up to 128gb would be nice. If rack mount then many nvme lanes would be great. A couple sata for os drives in a mirror seems good enough. Whether mini or rack mount, we need at least one pcie slot for networking, and many will want GPU as well.

 

[kapone]

IDK if hardware companies even care about low end consumers anymore.
Market for making money seems to be going to data center and AI. So we the end users are getting shafted with what's available or built for end users.
Software companies are moving to a subscription model for everything and soon OS will just be a small agent that connect something like mini pc with min specs up to a virtual desktop and services.

I would buy from a company that builds purpose built hardware for home lab or small business users.
I want something with 16+ power efficient cores/ Max Memory with ECC / IPMI / and more then 16 pcie lanes.
Maybe a company like minisforums might be able to build something like that if someone backs them?

It may or may not happen.

If it does though, you may baulk at the price. That’s just the harsh reality right now.

 

[kapone]

I would agree on the homelab specs. for the most part, 8c16t is a pretty good punch, and up to 128gb would be nice. If rack mount then many nvme lanes would be great. A couple sata for os drives in a mirror seems good enough. Whether mini or rack mount, we need at least one pcie slot for networking, and many will want GPU as well.

We‘re entering an era of… corporate control of hardware. You simply don’t have deep enough pockets to compete with that.

 

[mattventura]

If it does though, you may baulk at the price. That’s just the harsh reality right now.

That's the problem, yeah. Our best hope is stuff that can make the enterprise hardware usable in consumer platforms, like SXM to PCIe converter boards. Or maybe a Chinese company decides to start pulling chips off of boards and putting them onto new products as a way of recycling them.

 

[Greg_E]

We‘re entering an era of… corporate control of hardware. You simply don’t have deep enough pockets to compete with that.

I hope the old ways last long enough for me to retire, not that many years left for me.

 

[Stephan]

Have to wait for the Great AI Crash of 2028.

When API endpoints people take for granted today will go away, because operators run out of money. Datacenters will puke back out all those NVidia racks for cheap. Personally, looking forward to 8x H100 with 3D-printed cooling adapter and custom fan controller for less than 10K total. To run the coming 500B+ models. Downloaded DavidAU/Qwen3.5-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking · Hugging Face yesterday on test rig with 192G RAM and RTX A4000 16G VRAM. Very cool and really chatty about sensitive topics in China, but 2.5 tps on llama.cpp wow testing my patience.

Until 2028 and as a BTW, if you are investing in equity funds, recommend to check index methodologies of funds you hold. Because this and next year major AI-connected IPOs are happening. And what you don't want to be is exit liquidity for Elon and his pre-IPO arab investors: Nasdaq's Shame Considering revenues and depreciation true worth of those 1.75T is imho 100B all-in.

 

[ca3y6]

Irrespective of the AI boom, you still have the problem that all companies seem to be migrating their infrastructures to hyperscalers, and those don't really resell their hardware, and are using custom hardware anyway. So the pool of future used enterprise hardware supply is shrinking.

The only offsetting force is that the capacities (in term of RAM, cores, storage size, etc) increase probably faster than the pool of potential used enterprise hardware shrinks. Unless you want to run some crazy AI projects at home, it's not like a regular homelab can make any use of more than a couple of hundred cores, 1TB of RAM, or 1PB of storage.

 

[louie1961]

We‘re entering an era of… corporate control of hardware. You simply don’t have deep enough pockets to compete with that.

We have been in the era of corporate control of hardware for decades. Nothing new is going on, but the demand for certain flavors is changing.

 

[WANg]

Just few cents


Almost no one wants to buy those systems, funny reason i've heard "they didn't come with intel cpu, and its arm"


Are they? You can still slap 5090 on broadwell, and if model fits whole on the gpu - it will perform about the same. If we add llama.cpp into the mix we can split load between different gpu's without expensive nvlink, sxm boxes
(the split has full freedom, but obviously the matrix will run at speed of the slowest card.); There's no real need to run 100G models, they are often as smart as your 8-20G models. The problem is that the ai didn't make a lot of nv cards obsolete quickly... rising prices on old quadro/geforce 16-24-48G cards... this includes even some amd cards.


Doesn't this happen like every generation, the manufacturers find a way to raise them; Sure this time it rose a lot more - as scalping target is datacenter; but its nothing new.

in form of lot more expensive, the question is EoL and big companies flooding 2nd hand markets with lot of hardware. This too will happen again, all those datacenters will cycle hardware every 3-4 years; as it will be more profitable for them to run latest (just from power perspective). Once that happens we should expect enormous flood gates of cheap server grade hardware - if they really bought that much, and its not just scalping feast.

Eh, I don't think the issue lies with Intel versus ARM. Unlike most "popular" (i.e. Apple or Qualcomm) ARM implementations nVidia seemed to have made it straight-up UEFI, so you are not locked within a specific OS ecosystem, and unlike Qualcomm or the myriad of Rockchip or CP8180 miniPC clones, their chip is supposedly somewhat competitive perf-wise to x64 (Grace with the 10 X925/10 A725 perf-wise is like a Strix Halo but ARM based with much better compute support). Given enough soldered RAM and either a large spare inventory OR an active support contract I doubt the socketed versus soldered RAM debate is all that relevant.

While you can totally repurpose your Broadwell (DDR3L) to run that 5090, you can do the same with a Ryzen Renoir (DDR4) or even an Alder Lake i3 (DDR4 or 5), which will give you similar horsepower for less power use. RAM density goes up and power consumption goes down, and keeping those old battleaxes running will eventually catch up on power usage, especially if you are not pushing them. A 5090 is not really available at MSRP (already kinda ridiculous) unless you are lucky. nVidia made sure those cards are fought over and scalped to high orbit since day 1, and most people don’t buy 2200 dollar GPUs casually. Hell, I probably can’t even justify buying a 500 dollar AMD RDNA3 or a 400 dollar Intel Battlemage. The other issue with the RAM shortage is that fabs moved to higher margin HBM used within highly integrated/embedded applications, so even if the demand drops it's not like there are large amounts of normal RAM chips that can be quickly made into SODIMMs and DIMMs for addressing the current shortages. Frankenhardware and weird Chinese hacks maybe, but not much at volume that will send the prices diving.

Unless you end up picking up DGX mini-boxes at a fire sale, most of the racks coming out of data center fire sales will be too big, too hot and too hungry to be useful in your house for storing a few terabytes of photos+files, transcoding your video library or run a few home tasks like reading out your temp sensors or setting your lights on or off. Most home labs don't really do much other than idle 90% of the time anyways.
You can literally do that stuff on a Mac Mini or an Atom mini-box. For a good idea of what those Grace racks will be like, check out the Asrock BC250 box that used binned PS5 chips to mine crypto in .cn (too power hungry to be useful just idling, RDNA2 is not supported in ROCm and not super competitive in Vulkan for LLMs, and not enough HBM to make it worth much in general purpose computing) - that's just one module out of a rack or cabinet of many...compelling and possibly fun to have, but punishing if left for long.

 

[Greg_E]

With the common AI chassis being 8+u high, and over 2000 watts each, there's zero chance of them being common in homelab. I'm assuming people have watched Patrick's walkthrough of a few of these spaces, none of that gear is homelab friendly. Maybe Individual GPU, maybe processors and ram, but the rest seems highly specialized and expensive to operate.

 

[ca3y6]

Maybe Individual GPU, maybe processors and ram, but the rest seems highly specialized and expensive to operate.

and the SSDs!

 

[Greg_E]

Yes the drives may be possible, but I don't have any U.2 or U.3 capable servers anywhere, and it doesn't look good for finding used discounted stuff.

My first real lab was built from Dell 1u servers, every one of them had Facebook property tags on them. That specific recycle case may never happen again since Facebook is using custom designed hardware in their clusters now. A friend of mine is trying to get Kubernetes and Rancher running on them, labbing a situation that will come up at his work in the next few years, they are also looking at Harvester for the VMs in that same time frame. By then Harvester should be mature enough for their needs.

 

[T_Minus]

Yes the drives may be possible, but I don't have any U.2 or U.3 capable servers anywhere, and it doesn't look good for finding used discounted stuff.

My first real lab was built from Dell 1u servers, every one of them had Facebook property tags on them. That specific recycle case may never happen again since Facebook is using custom designed hardware in their clusters now. A friend of mine is trying to get Kubernetes and Rancher running on them, labbing a situation that will come up at his work in the next few years, they are also looking at Harvester for the VMs in that same time frame. By then Harvester should be mature enough for their needs.

I have some 24x u2 servers I'd sell currently with e5 v3\4 boards\chips but you can swap newer gen in LOL!

 

[T_Minus]

We have been in the era of corporate control of hardware for decades. Nothing new is going on, but the demand for certain flavors is changing.

Yes, and that new hardware is not homelab or even really "home" friendly... sure some of us have 240v and higher 3-phase at home, but I have 0 interest in running anything that power hungry at home for any length of time... maybe current frontier or higher level for short bursts and power down

in 2026 the smaller models are becoming more impressive, and if we can have a better orchestration layer or moe handling we can swap in\out niche models better and as needed

 

[kapone]

We have been in the era of corporate control of hardware for decades. Nothing new is going on, but the demand for certain flavors is changing.

Maybe I worded that poorly. What I really meant to say was that a lot of hardware of today is essentially designed for corporate use. These hardware would be difficult and impractical to run in a homelab scenario (as others have alluded to as well).

 

[kapone]

Until 2028 and as a BTW, if you are investing in equity funds, recommend to check index methodologies of funds you hold. Because this and next year major AI-connected IPOs are happening. And what you don't want to be is exit liquidity for Elon and his pre-IPO arab investors: Nasdaq's Shame Considering revenues and depreciation true worth of those 1.75T is imho 100B all-in.

I'm mostly a futures and options trader, with the occasional bond/CDS thrown in. If the S&P 500 mix changes, it'll be quite well telegraphed and...visible.