Hyper-V and file server upgrade time

waiter

New Member
Mar 18, 2021
6
0
1
Greetings

For a some years now I've been running a Hyper-V server at home for hobby and learning. It also serves as a file server. I use Storage Spaces so all the resiliency (mirroring, sometimes parity) is done in software not hardware. I plan to stick with Windows Server as that's what I want to explore. I think I'm ready to shift to all solid state storage and jump to Epyc.

My current rig is:
  • Desktop
  • Intel i5-2500k
  • 32 GB RAM
  • Samsung 840 EVO 1TB - boot and OS
  • 8x HGST HUH7280080ALE600 - storage
  • NVIDIA GeForce GTX 760 - periodically dabble with crypto mining

I'd like to upgrade and plan to keep the next server for a few years as well.

What I think I want is something like this...
  • Tower or rack
  • AMD EPYC 7313P
  • 128+ GB RAM (8x16GB with 8 available slots for future additions)
  • 2x 512GB NVMe SSD - boot and OS
  • 12x 240-960GB hotswap NVMe SSD - storage
  • 12x available hotswap NVMe SSD (for future additions)
  • (maybe) Crypto-decent GPU
Thoughts/input/recommendations?

I've been looking at Thinkmate's configurator a lot but (based on email with them) the configurator will allow you to spec a machine that is not viable (like let you pick U.2 drives when they are not supported with that chassis, for example). Also, I'd like to configure PCIe4 for at least the storage drives (but ideally all the drives) but Thinkmate doesn't have that option available yet.

Any other supplier to recommend?
 
Last edited:

Evan

Well-Known Member
Jan 6, 2016
3,252
559
113
I would vastly reduce the number of drives using larger sizes.
I assume Low idle power is not a target here because if it’s important I would looks at a lot of choices and change them.
 

waiter

New Member
Mar 18, 2021
6
0
1
Thanks for the input.

Even for a reduced drive count (6?) I'm still looking for good options for suppliers that provide a viable 7003 option with PCIe4 drives.

I would vastly reduce the number of drives using larger sizes.
That doesn't fit intent. Larger number of drive means more for striping, mirroring, etc. redundancy and different pools, etc.

As an example, with my existing rig when I had 4 drives it was barely useful for parity or 2 sets of mirrors. Moving to 5 allowed for a hot swap. Having 8 allowed for options like completing restructuring a 4 pool into a different 4 pool without having to move/copy the existing pool to a different computer during the transition.

Also, having room in the chassis for more drives allows for swapping in an entire new generation and testing the new drives in parallel with the old with less data loss risk. As an example, during one of the HDD upgrades I found that a brand new set of 4 HDDs were just incompatible and/or unreliable because of some firmware issue or somesuch. Fortunately, I didn't lose any data because the new drives starting flagging errors within the day of install. The current rig has physical room for 14 HDDs (but fewer SATA ports) which made the whole nightmare less painful.

Generally I've found over the last decade that for my testing that more independent devices the better.

I assume Low idle power is not a target here because if it’s important I would looks at a lot of choices and change them.
Elaborate?
 
Last edited:

Evan

Well-Known Member
Jan 6, 2016
3,252
559
113
ok so really it depends what you see the system doing, if power is somewhat important then;
- reducing drives decreases power but aside from that the number or proposed drives is pretty high for a home system, also say 2tb drives actually perform better generally that 1tn drives
- 7003 significantly increases the idle power consumption over 7002 , one reason its said that 7002 will remain on the market for smaller core counts and lower power.

If you see this beast working hard as indicated also by the GPU for mining then idle power consumption is of little relevance.

I just prefer a home server to be below say 50w idle as that’s what it does most of the day vs something like your looking at working hard using hundreds of watts.

I was only pointing this out in case you hadn’t considered it, worst is to invest into something then come back a few months later saying how costly it is to run, load, hot, etc
 

Evan

Well-Known Member
Jan 6, 2016
3,252
559
113
I would add 24 x NVMe drives is a high end 2u server usually. Nothing at all wrong with that just not what I see many people wanting for home :)

but the idea to usage all flash is exactly what I do, bulk storage on 3.84tb sata drives and for fast storage NVMe. Today the same could be done with 3.84tb NVMe for bulk storage as well but using nice lower power PCIe 3 drives that are 7mm usually or even M.2 maybe.
 

waiter

New Member
Mar 18, 2021
6
0
1
Great info across the aboard, Evan. Thanks!

Any recommendations on suppliers with useful configurators?
 

Evan

Well-Known Member
Jan 6, 2016
3,252
559
113
Well for that many NVMe disks the go to if you are willing to source branded drives etc is HPE or Dell 2U servers. I have good experience with supermicro as well but gigabyte and asrock rack etc should do just fine as well I only haven’t used them in any recent history.

Just about all of the options with that many MVMe disks will be datacenter rack mount loud servers.

I asked a question a while back about workstation as they are generally 4u/5u height if sideways and quieter, seems not much

 

waiter

New Member
Mar 18, 2021
6
0
1
Well for that many NVMe disks the go to if you are willing to source branded drives etc is HPE or Dell 2U servers.
Same issue with Lenovo:
Please note Lenovo ThinkSystem does not support 3rd party drives. Only Lenovo OEM drives are supported.

I asked a question a while back about workstation as they are generally 4u/5u height if sideways and quieter, seems not much
I ran into the same frustration. :|
 

Evan

Well-Known Member
Jan 6, 2016
3,252
559
113
And I just read about new 7003 systems from Lenovo, 1u and 2u only by the look of the quick information.
You can always build your own system but not with 24 NVMe slots really, if you don’t mind a smaller number it’s easier to use a own sources case, board, etc but when spending this much it’s also nice to have the support or a vendor (base vendor prices ok it’s just the drives and memory that’s a bit pricy)
 

waiter

New Member
Mar 18, 2021
6
0
1
This might be an interesting candidate for a slightly different direction than I was originally thinking...

Voracious Low-End Consolidation Tyan Transport CX GC68B8036-LE Review (servethehome.com)

Something like...
  • Rack @ 1U
  • CPU:
    • (A) AMD EPYC 7313P -- more cores, more idle consumption
    • or
    • (B) AMD EPYC 7232P -- fewer cores, less idle consumption
  • 128+ GB RAM (8x16GB with 8 available slots for future additions)
    • Any thoughts on 2933MHz vs. 3200MHz [@ Thinkmate]?
      • I'm used to expecting the faster at the same price to be better for gaming/desktops.
      • But for server, is lower power consumption the other consideration?
  • boot and OS
    • Micron 2300 M.2 PCIe 3.0 x4 NVMe
  • storage (fast)
    • 4x 3.84TB Micron 7300 PRO u.2 PCIe 3.0 x4 NVMe
  • storage (slow)
    • (A) 4x SATA SSD -- not as slow, lower capacity per $
    • or
    • (B) 4x SATA HDD -- really slow, higher capacity per $
  • (maybe) Crypto-decent GPU
 

Evan

Well-Known Member
Jan 6, 2016
3,252
559
113
Other than being a 1u loud solution if the price is good I really like the Tyan option.
With your sata bulk storage options are bit limited but hopefully enough, either really big data spinning disks if you need capacity else SSD if can live with less space but then use compression on whatever flavor is filesystem you plan, that can buy you more space and you have both ssd and plenty of cpu to support it.

If such a platform was in a bit bigger case and less noise I would probably buy one myself.
 

Evan

Well-Known Member
Jan 6, 2016
3,252
559
113
Well actual noise of commercial servers you can look at the specifications but now matter what unless it’s a really low power server anything that is 1u will be load, usually they have 40mm fans running at up to 15k rpm , like a jet engine taking off really. Smaller system generally = more noise.