Long time lurker and love this website! Looking for recommendations, tweaks, and advice on a new build that I am starting to work on. See below for all details:
Build’s Name:
VMWare Build
Operating System/ Storage Platform:
ESXI VMware vSphere with Operations Management Enterprise Plus v6.5
- Freenas VM (64GB RAM)
- Cisco VIRL VM (64GB RAM)
- GNS3 VM (8GB RAM)
- VCenter Server (10GB RAM)
- Windows 10 VM (4GB RAM)
- Windows 2016 Server VM (8GB RAM)
- PFSense VM (4GB RAM)
- (Misc/FUTURE VM's)
CPU:
1 x Intel Gold 6126
Motherboard:
1 x Supermicro X11SPL-F
Chassis:
1 x Supermicro CSE-846BE1C-R1K23B 24 Bay Storage Chassis
Drives:
24 x 8TB HGST NAS (Freenas RaidZ2 4 x VDEV, 6 drive per VDEV, and 1 x Pool for Storage)
1 x Supermicro SSD-DM032-SMCMVN1 32GB SATA DOM (ESXI Boot Drive)
2 x Intel Optane SSD 900P 480GB, AIC PCIe 4.0, 20nm, 3D XPoint (ESXI Datastore)
RAM:
6 x Samsung M393A4K40BB2-CTD 32GB DDR4-2666 Lp Ecc Reg Dimm
Add-in Cards:
1 x LSI SAS 9211-8i
1 x Intel Ethernet Server Adapter I350-T4V2
Power Supply:
2 x PWS-1K23A-1R (1200W Power Supply)
CPU Heatsink:
1 x Supermicro (SNK-P0068APS4) 2U Active CPU Heat Sink Socket LGA3647-0
Cables:
2 x Supermicro (CBL-SAST-0508-02) Internal MiniSAS o MiniSAS HD 50cm Cable
Usage Profile:
VMWare ESXI to consolidate existing VMWare and Storage servers into 1 box. Below are the main specs of the existing servers:
Existing VMWare ESXI Server
-Intel Xeon E3-1270v2
-Supermicro MBD-X9SCM-F-O
-Kingston 32GB (4 x 8GB) DDR3 SDRAM ECC Unbuffered 1333
-16GB USB Drive (ESXI Boot)
-250GB OCZ Vertex 4 (ESXI Host)
-1TB WD Black HDD (Datastore)
Existing Freenas Storage Server
-Intel Xeon E3-1230v1
-Supermicro MBD-X9SCM-F-O
-Kingston 32GB (4 x 8GB) DDR3 SDRAM ECC Unbuffered 1333
-20 x WD Red 3TB NAS running RaidZ2 with 2 x 10 vdev in 1 x storage pool
-2 x IBM M1015 (Flashed to IT)
-Norco 4220
Comments:
So I will start off with where I have issues or require recommendations in choosing hardware.
CPU/Motherboard/Compatibility
So between the Intel W-2155 or Intel Gold 6146 or AMD Epyc 7401P. For the price, the Intel W-2155 or AMD Epyc 7401P are the best. The issue I have with the Intel W-2155 is the Supermicro motherboards don't have much in PCI-E slots. This would hinder further upgrades and expansion of cards which might be a deal breaker. The AMD Epyc 7401P has quite a few expansion slots but I worry about compatibility and loss of some features. I will be running this in a VM environment, and as such need to make sure that I am able to do pass through of hardware (HBA cards to Freenas VM). Also worried about not being able to utilize Intel's AES-NI that I currently use for PFSense and encryption accellaration. The Intel Gold 6146 is the safe bet but also much more expensive.
HBA Choices
I am out of touch on what the latest recommendations for these cards are. I will require the ability to be able to do passthrough of the hardware to Freenas in a VM environment. As such, I know that I will either require 3 x 8 port or 2 x 16 port cards. Having SAS3 would be nice too for future upgrades but if not a big deal.
Cases
Currently have a Norco 4220 and am familiar with the case. It has served me well over the years. Now I know that Supermicro cases are better built that include dual power supplies and also offer expansion options. Basically looking for some general recommendations and even the possibility of upgrade SAS2 expanders to SAS3 if I wanted to. Will require at least 20 drive bays but might consider more options.
RAM
This will have to be determined a little later and will depend on which CPU/Motherboard I choose.
Edit 1: All comments/concerns have been addressed so far.
Other:
This really is a culmination of various topics but thought it would be best suited in this forum. Feel free to move it if needed.
Build’s Name:
VMWare Build
Operating System/ Storage Platform:
ESXI VMware vSphere with Operations Management Enterprise Plus v6.5
- Freenas VM (64GB RAM)
- Cisco VIRL VM (64GB RAM)
- GNS3 VM (8GB RAM)
- VCenter Server (10GB RAM)
- Windows 10 VM (4GB RAM)
- Windows 2016 Server VM (8GB RAM)
- PFSense VM (4GB RAM)
- (Misc/FUTURE VM's)
CPU:
1 x Intel Gold 6126
Motherboard:
1 x Supermicro X11SPL-F
Chassis:
1 x Supermicro CSE-846BE1C-R1K23B 24 Bay Storage Chassis
Drives:
24 x 8TB HGST NAS (Freenas RaidZ2 4 x VDEV, 6 drive per VDEV, and 1 x Pool for Storage)
1 x Supermicro SSD-DM032-SMCMVN1 32GB SATA DOM (ESXI Boot Drive)
2 x Intel Optane SSD 900P 480GB, AIC PCIe 4.0, 20nm, 3D XPoint (ESXI Datastore)
RAM:
6 x Samsung M393A4K40BB2-CTD 32GB DDR4-2666 Lp Ecc Reg Dimm
Add-in Cards:
1 x LSI SAS 9211-8i
1 x Intel Ethernet Server Adapter I350-T4V2
Power Supply:
2 x PWS-1K23A-1R (1200W Power Supply)
CPU Heatsink:
1 x Supermicro (SNK-P0068APS4) 2U Active CPU Heat Sink Socket LGA3647-0
Cables:
2 x Supermicro (CBL-SAST-0508-02) Internal MiniSAS o MiniSAS HD 50cm Cable
Usage Profile:
VMWare ESXI to consolidate existing VMWare and Storage servers into 1 box. Below are the main specs of the existing servers:
Existing VMWare ESXI Server
-Intel Xeon E3-1270v2
-Supermicro MBD-X9SCM-F-O
-Kingston 32GB (4 x 8GB) DDR3 SDRAM ECC Unbuffered 1333
-16GB USB Drive (ESXI Boot)
-250GB OCZ Vertex 4 (ESXI Host)
-1TB WD Black HDD (Datastore)
Existing Freenas Storage Server
-Intel Xeon E3-1230v1
-Supermicro MBD-X9SCM-F-O
-Kingston 32GB (4 x 8GB) DDR3 SDRAM ECC Unbuffered 1333
-20 x WD Red 3TB NAS running RaidZ2 with 2 x 10 vdev in 1 x storage pool
-2 x IBM M1015 (Flashed to IT)
-Norco 4220
Comments:
CPU/Motherboard/Compatibility
So between the Intel W-2155 or Intel Gold 6146 or AMD Epyc 7401P. For the price, the Intel W-2155 or AMD Epyc 7401P are the best. The issue I have with the Intel W-2155 is the Supermicro motherboards don't have much in PCI-E slots. This would hinder further upgrades and expansion of cards which might be a deal breaker. The AMD Epyc 7401P has quite a few expansion slots but I worry about compatibility and loss of some features. I will be running this in a VM environment, and as such need to make sure that I am able to do pass through of hardware (HBA cards to Freenas VM). Also worried about not being able to utilize Intel's AES-NI that I currently use for PFSense and encryption accellaration. The Intel Gold 6146 is the safe bet but also much more expensive.
HBA Choices
I am out of touch on what the latest recommendations for these cards are. I will require the ability to be able to do passthrough of the hardware to Freenas in a VM environment. As such, I know that I will either require 3 x 8 port or 2 x 16 port cards. Having SAS3 would be nice too for future upgrades but if not a big deal.
Cases
Currently have a Norco 4220 and am familiar with the case. It has served me well over the years. Now I know that Supermicro cases are better built that include dual power supplies and also offer expansion options. Basically looking for some general recommendations and even the possibility of upgrade SAS2 expanders to SAS3 if I wanted to. Will require at least 20 drive bays but might consider more options.
RAM
This will have to be determined a little later and will depend on which CPU/Motherboard I choose.
Edit 1: All comments/concerns have been addressed so far.
Other:
This really is a culmination of various topics but thought it would be best suited in this forum. Feel free to move it if needed.
Last edited: