Help me find: Bad ass VM server

[AJXCR]

The more I'm thinking the more I like this idea... because I can add an adapter to existing servers and get storage for those too all on the storage host. Nothing I've done before, so I'm a newb at it but I like the idea of this.

Personally, this is the route I've taken using a disk shelf. An array of 8x HGST HE8 6TB SAS3 drives reads/writes surprisingly fast (at least I was extremely impressed)...

 

[Evan]

Personally, this is the route I've taken using a disk shelf. An array of 8x HGST HE8 6TB SAS3 drives reads/writes surprisingly fast (at least I was extremely impressed)...

Yep the HE8’s are pretty good for spinning disks.
I took a different route to bulk storage and that was just sata SSD all the way but needs a little more modest and I purchased when ssd’s were cheaper, bit pricey to do that today but as @AJXCR has also offered there is some value to be has with larger SAS SSD also.

 

[El Chorizo]

Personally, this is the route I've taken using a disk shelf. An array of 8x HGST HE8 6TB SAS3 drives reads/writes surprisingly fast (at least I was extremely impressed)...

What kind of chassis/interconnect are you using?

 

[whitey]

If it's just for learning/playing than I would build two systems, one storage node and one vm host.

Storage node with 100gbe/edr infiniband + optane ssd(s) for slog (900p or p4800x), max 128gb ddr3/4
Vm host with 100gbe/edr infiniband and two cpu (8+ cores & 2+ ghz each)

Put everthing that you're not spending on hdds/ssds (capacity) in ram for the vm host.

Imma' go out on a limb here and say that 100GbE is overkill w/in this budget, 10GbE would work but w/ 40GbE being soo price competive to 10GbE why not go for 40GbE but 100...nah, you'll blow 1/2 if not all budget on ethernet fabric before you even get out of the gate unless you just go 2 node and direct connect 100GbE nics and that still is not cheap.

 

[AJXCR]

Yep the HE8’s are pretty good for spinning disks.
I took a different route to bulk storage and that was just sata SSD all the way but needs a little more modest and I purchased when ssd’s were cheaper, bit pricey to do that today but as @AJXCR has also offered there is some value to be has with larger SAS SSD also.

Don't get me wrong, I've got no shortage of solid state storage.. But the (only) spinning disks I have went into a disk shelf.

 

[AJXCR]

What kind of chassis/interconnect are you using?

Chassis: CSE-219 & 847.. this has been changed around a lot recently while playing with various gear
External HBA's- LSI 9305-16E's
Internal to External adapters- AOM-SAS3-16i16e-LP's
JBOD board: CSE-PTJBOD-CB3
Ext SAS3 Cables: CBL-SAST-0573's
Internal SAS3 Cables: CBL-SAST-0532's