Hi,
I decided to downscale my homelab a bit from 3 - down to just one machine.
I did the downscale mostly because I was too ambitious about what I wanted to use the lab for and also to save a little power, since not having two switches and two servers running does add up.
My previous setup was:
3 machines with:
To:
1 Machine with:
It now runs my FreeNAS with the NVME drives passed through and all my previous VM's.
And idles around 150-200w with all VM's powered up doing their stuff (which is not a lot mostly)
So my not so scientific guess is that I have probably saved around 200w/h, which with the crazy power prices we have in Denmark amounts to around 660 USD per year in power savings.
Conversion was mostly smooth, by just migrating the VMS to the new server, exporting the FreeNAS configuration, creating a new FreeNAS VM, pass through all the hardware to the new FreeNAS VM (with a PITA panic attack, when I saw nvme msi errors in FreeNAS - which took quite some googling to figure out, but got solved by: https://forums.servethehome.com/index.php?threads/optane-slog-passthrough-or-alternatives.27596/ (hw.pci.honor_msi_blacklist=0) in FreeNAS - everything else did not work for me.
When FreeNAS was up and running, I just imported the config from the old server, changed the network assignments and voila everything just worked.
Good news is that I will have 3 motherboards + other stuff to sell
https://forums.servethehome.com/index.php?threads/lab-downscale.30504 (FS thread)
tada
I decided to downscale my homelab a bit from 3 - down to just one machine.
I did the downscale mostly because I was too ambitious about what I wanted to use the lab for and also to save a little power, since not having two switches and two servers running does add up.
My previous setup was:
3 machines with:
Code:
SuperMicro X10SRi-F + 1 x Xeon e5-2673v3 + 256GB RDIMM/LRDIMM
With combined equipment spread across the machines:
FreeNAS:
2 x Intel P4600 2TB
2 x Intel P4510 2TB
1 x Mellanox ConnectX-3/-Pro
1 x SuperMicro NVME controller
1 x USB Boot stick
VM1:
2 x Intel P3700 400GB
1 x Mellanox ConnectX-3/-Pro
1 x USB boot stick
VM2:
1x Intel Optane 900P 280GB
1xIntel DC S4600 480GB
1xMellanox ConnectX-3/-Pro
1 x USB boot stick
Switches:
1xMellanox SX6018 Switch
1xBrocade ICX 6450-24P
1 Machine with:
Code:
X10DRi-T + 2xXeon e5-2673v3, 512GB LRDIMM
2xIntel P4600 2TB
2xIntel P4510 2TB
1x Intel Optane 900P 280GB
1xIntel DC S4600 SATA disk 960GB
1xSuperMicro NVME controller
And idles around 150-200w with all VM's powered up doing their stuff (which is not a lot mostly)
So my not so scientific guess is that I have probably saved around 200w/h, which with the crazy power prices we have in Denmark amounts to around 660 USD per year in power savings.
Conversion was mostly smooth, by just migrating the VMS to the new server, exporting the FreeNAS configuration, creating a new FreeNAS VM, pass through all the hardware to the new FreeNAS VM (with a PITA panic attack, when I saw nvme msi errors in FreeNAS - which took quite some googling to figure out, but got solved by: https://forums.servethehome.com/index.php?threads/optane-slog-passthrough-or-alternatives.27596/ (hw.pci.honor_msi_blacklist=0) in FreeNAS - everything else did not work for me.
When FreeNAS was up and running, I just imported the config from the old server, changed the network assignments and voila everything just worked.
Good news is that I will have 3 motherboards + other stuff to sell
https://forums.servethehome.com/index.php?threads/lab-downscale.30504 (FS thread)
tada
Last edited: