New HomeLab server

[luckyz0r]

Hello,

I have a Microserver Gen8 - it's runing ESXI with Pfsense, Home assistant, OMV6 and RHEL9 for containers
E3-1230 v2
16 RAM ECC (it's the maximum)
4x4TB WD RED NAS

I'm running everything in one and only server, if something goes wrong, well no internet, nothing.
I wanted to buy a 2nd server to split a bit all of this.

keep the Gen8 but with TRUENAS or something like that for storage + VM for docker (filebrowser, qbtorrent,photoprism, etcc) and one more VM for tests

the 2nd server would be for proxmox - pfsense + home assistant + vm for containers used for networking (pihole+unifi controller, etc...)
Searching for something energy efficient and where I can add some NICs for pfsense

what do you think about this design? should I change anything?

thanks for your help

 

[CyklonDX]

i presume this is meant to be power efficient, quiet, not overheating and small?

The ram limitations on this box is terrible, cpu is ... not great. Its fine for NAS.


In most cases if you don't have a lot of hardware, i would just get better workstation for yourself.
Installl unraid or Truenas there, and run everything from your workstation. You can get decently cheap workstation hardware a T7910 (haswell-broadwell) platform that can hold over 1TB of ram, 2x CPU's if needed, and plenty of pcie ports. It typically offers 4x 3.5" disks hotswap, and 4x 2.5" disks hotswap. The cpu's are not that expensive either you can get e5-2690v4 (14c 2.6-3.5GHz) for 100 usd. 256GB of ddr4 ecc reg 2400 can go for 350-400 usd.
Build quality is good.

(ebay - aprox costs in usd - but you don't have to start high end... you can work your way up)
T7910 ( 350-450)
256GB DDR4 2400MHz (360)
2x e5-2690v4 (200)
1-4 GPU's (0-infinite k)
4x 8TB HGST (248) ~ setup zfs raidz1 29TB
4x 1TB Crucial MX500 (350) ~ raidz1 around 3.6TB
2x nvme for L2ARC for zfs (100-400 usd)
10gig network card (100 usd)


Your play-thing-desktop would be just a kvm with a GPU, and everything could be running in dockers, pfsense, home crap, anything you desire. While still running your desktop quite isolated.

 

[bigfellasdad]

Cyklondx, really? From a gen8 microserver to your suggestion. Wow.
The obvious answer is another gen8 microserverl to give redundancy is it not?

They are getting long in the teeth though I must admit. There are plenty of sff quiet and very power efficient machines our there. What can you see yourself needing?

Just a thought but a gen 6 xeon e3 server with up to 128gb and all around faster that your gen8 can be bought cheaply and idle at sub 20w with extra nice and a couple of drives

 

[CyklonDX]

yes instead of running 2 microboxes with limited functionality, + his personal computer/s. This would just put single box in place of them all.

 

[bigfellasdad]

But it would be running all of the time, even idle that setup is going to be high power draw.

I'm a bit anal on power usage now, I've gone green in my old age.
My nirvana is a very low power 24/7 server running my must have services including openwrt, haos, home samba share and a management ansible vm. It draws 8w at idle with 2x2.5gb usb nics and a 2tb 2.5ssd and 1tb nvme. I'm happy with that, m920q. My play lab is a hp z240 with lots of ram and strong cpu and big storage. I use vpro to power it up only when needed. If I need to work on my m920q I just spin up the openwrt on the other server and all is good. One cable flip and a few clicks.

It's very low power and capacity on demand. The z240 draws 2/3w when not powered on due to vpro nic being up

My pc is a full on gaming rig, but it's only on maybe 2hrs a day at most so better to have that powered off when not in use

 

[CyklonDX]

At idle on similar workstation setup to t7910, i'm using only 340W on UPS with following:

2x e5-2697v2, 256GB ram 50% util, 8 disks (4x sas 8tb, 4x ssd's), NV A4000, NV 2080, nvme card x16 (4nvmes), dual 10gig nic, external fan controller, switch, hdmi switch, poe camera switch and jbod storage with 8x8TB disks.

At full load during summer it can use 1.44kW though on that UPS.

Thus if you went more conservative you could get away with 40-80W idle. Depending what hardware you pick. Idle power usage can get really low, and some gpu's sport all kinds of power saving modes when not in use. UPS also can result in lower actual electricity consumption.

A decent size cheap solar panel could potentially reduce during the day the usage to 20-30W if used with ups.

// As note, i don't really trust what system states what its is power draw; i use those on my walls to see how much power i actually use and compare month/by/month.

https://www.amazon.com/dp/B079JWGS92

I recommend hooking up your stuff to similar, and checking over a month period if its worth or not. Many would be surprised.

In my personal tests at home 'green' hardware/power saving is the most misleading in computing... you slow things down so it takes more time to process things, in the end eating up more time power, and providing worse overall experience; (then again it depends on usage.)

 

[oneplane]

You need to put network/internet on one of these: https://forums.servethehome.com/ind...lake-quad-i225v-mini-pc-report.36699/page-110 (or the i226v version).

As for the other stuff: a NUC could handle that these days. Most important thing is probably taking inventory of what is important. Maybe all you end up with is a sinology NAS with a Ryzen embedded CPU. Does all your transcoding and containers and storage needs. Very power efficient, and coupled with one of the Jasper Lake network boxes you'd get triple the performance for less than half the energy usage.

Whichever way you go, a microserver won't be the way ;-)

 

[bigfellasdad]

The perspective here is, today the op has a gen8 microserver with 4 drives, 18w idle 80w full load maybe. He seems happy with what he has. That's why I. Suggesting similar power usage levels but more up to date and far more capacity... cheaply.
I use a watt mater and smart power plugs linked in to ha btw

as oneplace says, the gen8, although an ace bit of kit..... is pretty slow, strangely its hindered mostly by IO and ram expansion. It only has 2xsata3 and 2xsata2 ports, with an additional sata3 for ODD/or additional ssd.. Its also clobbered by the b140i sata controller. More modern kit is so much better. Id suggest what ever you end up with for docker/vm's, ensure you can put them on nvme SSD's, and use your nas as backups and likely your router/haos, you have 2 nics to play with wan/lan are covered..... its nice having an ilo as well too, if the microserver goes down you can connect over ilo to fix as long as you have the professional ilo license (check ebay for that!). Install proxmox on it too, you can then migrate VMs between your new proxmox box and even use the proxmox backup service to full effect.

 

[luckyz0r]

Cyklondx, really? From a gen8 microserver to your suggestion. Wow.
The obvious answer is another gen8 microserverl to give redundancy is it not?

They are getting long in the teeth though I must admit. There are plenty of sff quiet and very power efficient machines our there. What can you see yourself needing?

Just a thought but a gen 6 xeon e3 server with up to 128gb and all around faster that your gen8 can be bought cheaply and idle at sub 20w with extra nice and a couple of drives

can you give me some examples?

The perspective here is, today the op has a gen8 microserver with 4 drives, 18w idle 80w full load maybe. He seems happy with what he has. That's why I. Suggesting similar power usage levels but more up to date and far more capacity... cheaply.
I use a watt mater and smart power plugs linked in to ha btw

as oneplace says, the gen8, although an ace bit of kit..... is pretty slow, strangely its hindered mostly by IO and ram expansion. It only has 2xsata3 and 2xsata2 ports, with an additional sata3 for ODD/or additional ssd.. Its also clobbered by the b140i sata controller. More modern kit is so much better. Id suggest what ever you end up with for docker/vm's, ensure you can put them on nvme SSD's, and use your nas as backups and likely your router/haos, you have 2 nics to play with wan/lan are covered..... its nice having an ilo as well too, if the microserver goes down you can connect over ilo to fix as long as you have the professional ilo license (check ebay for that!). Install proxmox on it too, you can then migrate VMs between your new proxmox box and even use the proxmox backup service to full effect.

My Gen8 is running on my garage, sound is not an issue, however if I can have something good without use 400w in idle it would be great.

I was thinking on keep the gen8 as NAS, but I can use it for the pfsense + home assistant with SSD's for example, I can remove the HP H220 IT-mode LSI 9205-8I, I bought (for the passthrough of the 4 disks to the OMV) and install a pcie card with more NICS for the pfsense VM.

what should I chose for NAS + VM's? any links? configs?
thanks

 

[bigfellasdad]

can you give me some examples?

For a gen8 replacement or 2nd VM box?
Ive got 2 HP Z240's but SFF models (one is gathering dust, if you are in the uk you can have it ), they have 2x3.5 and 2x2.5 slots, the workstation can take more drives so likely more suitable. One has an i3 gen 6 and the other an E3-1245v6 and idle sub 20w with 2x3.5 HDDs, 2x16gb ddr4 (i think 64gb max, but it might be 128gb, and 1tb NVME drive, also Cheap as chips and silent.
Proxmox installed, sata drives in a pool and shared out with samba == NAS, vme drive used as boot and VM drive, backed up to my pool.
maybe HP Z240 Tower Workstation Intel Core i7-6700 3.4GHz 16GB RAM - NO HDD - NO OS | eBay would suit.. Gen6 CPUs are very good for power too.

I was thinking on keep the gen8 as NAS, but I can use it for the pfsense + home assistant with SSD's for example, I can remove the HP H220 IT-mode LSI 9205-8I, I bought (for the passthrough of the 4 disks to the OMV) and install a pcie card with more NICS for the pfsense VM.

I dont really have a need for more than my 2tb share, so my setup is easier than yours. Ive actually just watched a link provided in another thread, it may be perfect for you Building a Power Efficient Home Server! - YouTube . You need to ask yourself what you need now and maybe in the next year or 2. Based upon that im sure we can all help more specifically.

HTH give some ideas.

 

[bigfellasdad]

one other tip, which ever box you chose to upgrade to, make sure the cpu has on board graphics, you dont want to install a graphics card that burns power if you very rarely need it.

 

[CyklonDX]

one other tip, which ever box you chose to upgrade to, make sure the cpu has on board graphics, you dont want to install a graphics card that burns power if you very rarely need it.

if the graphics has L1 substates its going to be quite the same when idle. (tho i don't believe he needs to worry about that at all as his microserver already comes with integrated matrox, and hp ilo card.)

 

[luckyz0r]

for the pfsense router (I really wanted to have a dedicated hardware for this)
I bought a old fugitsu S920 for35€, I will put a 2 nic port 1Gb (i will change this for a 2.5 or 10gb nics)

I really love my HPgen8, but it lacks of "space" to upgrade, I wanted to create a windows VM (for some tests) and he really strugles, for the rest it is really great.

I will use my gen8 with proxmox + Home assistant + RHEL9 for containers related to network (pihole, unifi controler, etc..), I will use 2 SSDs 1TB install proxmox use it for the VM's too

now I wanted a solution for my NAS, I saw a R730 in the 600€ range (i've already have 4x4TB NAS RED) but I saw they are very power ungry and i think I can't use the raid card so I need to buy a pcie sata to connect the disks?

in a 600€ range without the disks, any diy server would be better? I don't care about the noise,size, rack or not rack (don't have a rack so a "desktop" case is ok)
something I can upgrade later, start with for example 32gb ECC ram

any ideas?
Thank you

 

[CyklonDX]

saw they are very power hungry

depends how you see power hungry. A decked out R730xd filled with 12x3.5" sas disks takes ~266W on idle, and 790W on full load (cpu, ram, disks)
here's from one i have at work (its mostly used for log storage)

 

[oneplane]

266W can be a bit much if you consider other systems take only 10% of that power when idle. It also really depends on your compute power needs.

 

[luckyz0r]

depends how you see power hungry. A decked out R730xd filled with 12x3.5" sas disks takes ~266W on idle, and 790W on full load (cpu, ram, disks)
here's from one i have at work (its mostly used for log storage)

View attachment 26539

For now all my network (3 switchs, 3 unifi APs, isp modem),the microserver gen8 and 1 camera, I have a constant consuption of 70/80w.
250w+ only for the server is really a lot.
In EU the price of the electricity is very, very high so I would say, each watt counts.

I don't need something as powerful as the r730 but something more powerful than my Microserver.

 

[CyklonDX]

keep a note this was a decked out r730.
(2x e5-2690v4's, 512GB DDR4 2400, and 12x8tb sas disks this alone is likely eating 100W on idle)
A sleazy box, with single cpu, LP dimms, ssd's, and balanced/or perf on demand fan curve -> i'm sure you can get around ~80W of utilization when idle.

 

[NPS]

if the graphics has L1 substates its going to be quite the same when idle.

You really live in another world concerning what is "quite the same". A GPU drawing less than 5W idle would be hard to find but 5W can be quite a lot in a reasonably dimensoned home server for the use cases. Remember its only for management. Even modern IPMI draws less. A iGPU draws less than you can measure.
I would recommend something like intel LGA1151. Personally I like the v2 variant of that socket. C242/C246 chipset with i3-9100 is hard to beat for a reasonably priced home server with low power consumption if you want to have ECC. And as a bonus you can get quite some PCIe lanes if you choose the right board. If you don't need ECC there's uncountable options... If you need more CPU (I doubt you will) or more that 128GB RAM, power draw for the base system (CPU, MB, RAM) will at least double but in most cases tripple (still single socket).

 

[luckyz0r]

You really live in another world concerning what is "quite the same". A GPU drawing less than 5W idle would be hard to find but 5W can be quite a lot in a reasonably dimensoned home server for the use cases. Remember its only for management. Even modern IPMI draws less. A iGPU draws less than you can measure.
I would recommend something like intel LGA1151. Personally I like the v2 variant of that socket. C242/C246 chipset with i3-9100 is hard to beat for a reasonably priced home server with low power consumption if you want to have ECC. And as a bonus you can get quite some PCIe lanes if you choose the right board. If you don't need ECC there's uncountable options... If you need more CPU (I doubt you will) or more that 128GB RAM, power draw for the base system (CPU, MB, RAM) will at least double but in most cases tripple (still single socket).

Could this work for my TrueNAS + plex?
Only need max 2 transcoding (the transcoding is because of internet limitation, my upload at home is shit)

330€ with shipping:

Dell T330 Tower Server
i3-6100
8GB RAM DDR4 (will change this for ddr3 1600 ECC-4x16go ram ddr3L Ecc Samsung / 64Gb PC3L 1600mHz / PC 12800R - will this be ok? )
Standard 8x Hot Plug 3.5" LFF Backplane (SAS/SATA)
DELL PERC H730
DELL PERC S130 (embedded)
1x 495 Watt (can install a sencod one latter)
DELL iDRAC8 Remote Management (Enterprise Version)

Thanks for your help

 

[NPS]

I'm personally not familiar with the Dell systems, but RAID-controllers like the PERC H730 are a no-go für ZFS (TrueNAS) you need chipset-integrated ports or an HBA. Maybe you can remove the RAID-card and use the ports of the C246, but I don't know if the board/backplane support this.

If you don't have a specific need for a second PSU, I would opt for using just one, because of power consumption.

I have no experience with plex so can not comment on this.

All together this T330 locks interesting but it is a high end system in terms of features (redundant PSU, RAID card, hot-plug backplane, IPMI). Maybe someone can comment on the power consumption of such a system. My experience is limited to a Fujitsu/Kontron board without IPMI (very high quality and low power consumption) and a Supermicro X11SCL-iF with IPMI (not that much additional power consumption).