Refreshing the ESXi server

Discussion in 'DIY Server and Workstation Builds' started by pc-tecky, May 16, 2018.

  1. pc-tecky

    pc-tecky Member

    Joined:
    May 1, 2013
    Messages:
    154
    Likes Received:
    21
    Build’s Name: WorkHorseII
    OS: ESXi (or HyperV) to host Windows Server, Windows, Linux, BSD, FreeNAS, pfSense, unTangled, etc.
    CPU: 2x Intel Xeon E5-2670 (V1)
    Motherboard: Intel S2600CP2J
    Chassis: Intel P4216xxMHGR
    RAM: 128GB (16 x 8GB)
    Drives: Various; Seagate 4TB for FreeNAS
    Power Supply: Default per chassis (redundant 750W)

    I'm looking to refresh, clean up, or move on. This server sadly has sat unused and disconnected. A few things to resolve:

    1) maybe upgrade to E5-26?? V2 CPUs

    2) Drive storage options (SuperMicro M28SAB-OEM, Intel FUP8X25HSDKS (hack/mod), Intel FUP8X25HSDKO (eol), etc.) This will determine how I configure the box. 1x,2x, or 3x SAS controllers, or use the Intel "240" or "360" Expander? That in turn determines which cables I use and where I route them.

    3) PDU Auxillary PCIe Power to Graphics Cards (custom cables?)

    4) LSI 2008 detects that the 5x 4TB Seagate HDDs are alive, but not their capacities?!?

    Just got ESXi 6.7 installed to 32GB Lexar USB flash.

    Wow, Java is the framework for accessing IPMI on the Intel S2600CP2J? Nothing at all like Supermico's IPMI that I'm used too on several boards. :(

    Have ~24-28GB free on the USB boot device, but I need a different drive volume for datastorage?!? o_O Yeah, sorry, it's been long while. I thought I did just that with 2x 1.0 TB HDDs with ESXi 5.0-5.5 hardware - maybe mistaken? :( Also, trying to figure out the whole pass through and possible LSI 2008 limitations to drive capacity - I can see 5x Seagate HDDs from within ESXi, but they just don't have any capacity displayed. Is this an issue with sector sizes - from 512b to 4K as I recall, or a drive capacity issue?

    Add-On Cards Installed:
    1x 6-port 1.0 GbE card
    2x LSI 8i 6G SAS controllers (IT mode)
    1x Intel "360" SAS Expander (tight - no cables) designed to be single or dual ported? It there a manual for the Intel "360" SAS Expander? Where do the I2C cables go?

    This is what I got:

    USB Flash Drives: various USB 2.0 and USB 3.0/3.1 flash drives upto 128GB; likely to use the "nano" (Sandisk USB 32GB Fit or similar) - using 32GB Lexar (purple plastic slider) USB flash drive.

    Drives: (mix'n'match and move around if needed)
    256GB Toshiba Q Pro series SSD (atm, Win10 boot drive)
    480GB SandDisk Ultra II SSD (floating)
    500GB Samsung 850EVO SSD (floating, Raspberry Pi)
    others I'm sure..

    ~10x 4.0TB 2.5" 15mm (shucked or can be) Seagate HDD

    Add-on Cards Available:
    1x nVidia Quadro 4000 - the one available PCIe x16 slot is limited to half sized cards (otherwise the RAM interferes)
    2x Infiband dual-port 40GB controllers (no cables)
    1x Intel 8i 6G SAS (B0 stepping) SAS controller (locked/fixed to Intel S2600CP - NO cross flashing)
    1x Intel "240" SAS Expander
    (various SAS to SAS and SAS to SATA cables)

    What I still don't know with both Intel Expanders is: 1) Will 2x miniSAS double bandwidth performance (like they do with HP Expanders)? 2) Any known issues or bottlenecks using the Intel SAS Expander(s) with SATA drives?

    I'm wanting to keep the "lower" 16 bays strictly for FreeNAS drives (if I can). As I still have 3 open 5.25" bays, I'm looking for drive bay options. Do I try to low bid on an Intel FUP8X25HSDKS Spare 8x2.5" Hot-swap Drive Bay to fit into 2x 5.25" bays to hack and modify? Or find some other IcyDock Solution (like the one listed on the front page the other day)?

    In my digging, I discovered that PCIe Slot5 is on CPU socket 2, all the others are on CPU socket 1. I have both socket's occupied - Does this even matter for a VM like pfSense/unTangled, etc??

    -- Near future upgrade options:
    $359.99 - Matched Pair Intel Xeon E5-2680 V2 SR1A6 2.80GHz 25MB 10 Core LGA2011 CPU
    $279.99 - Samsung 32GB 4Rx4 PC3L-10600L-09-11-C0 RAM
    $52.99 - Kingston KTD-PE318/16G 16GB Memory RAM PC3-14900R
    $49.99 - Myricom 10G-PCIE-8A-S [single port] Network Adapter
    $52.99 - Myricom 10G-PCIE-8AL-S [single port] Network Adapter
    $104.99 - Myricom 10G-PCIE2-8B2-2S [dual port] Network Adapter
    $34.99 - Chelsio CC2-N320E-SR Dual Port Network Adapter Card
    $44.99 - IBM 5769 Chelsio 10GB PCI-e Single Port Network Adapter 46K7897
     
    #1
  2. nthu9280

    nthu9280 Active Member

    Joined:
    Feb 3, 2016
    Messages:
    768
    Likes Received:
    170
    Drive bay options:

    Try If you can find a good deal on a2u8x25s3<*>
    Those are designed for 2u rackmount servers and should fit in the 2x 5.25 bay space. Securing may be a challenge.

    I think newer variants can support NVME also.

    You have already listed SM and ICYDOCK options.

    Intel SAS HBA - with 2308 C0 chip.
    Though it's locked to Intel MB, you can flash to LSI 9205 P20.07 just not 9207

    PCIe power cables.
    Check kalleyomalley for P4000L or P4000M electric accessories kit. Been about 2 years but I think they accepted very reasonable offer for the kit.

    If not PM me and I can check my parts bin for spares.








    Sent from my Nexus 6 using Tapatalk
     
    #2
Similar Threads: Refreshing ESXi
Forum Title Date
DIY Server and Workstation Builds AIO ESXi/Napp-it advice requested Apr 4, 2018
DIY Server and Workstation Builds ESXi Ryzen build Jan 30, 2018
DIY Server and Workstation Builds Silent ESXI Host Dec 12, 2017
DIY Server and Workstation Builds Small/cheap ESXi homelab server Oct 28, 2017
DIY Server and Workstation Builds Small form factor ESXi with 64GB+ RAM? Oct 17, 2017

Share This Page