What did I miss? Intel S2600WT2

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
Alright guys, I'm in the process of assembling a relatively high spec Freenas box using a set of E5-4655V3's and all flash storage. Due to the current lack of PCIe x16 NVMe HBA availability, I have several unused 1.2TB 2.5" Intel 750's laying around which I would like to incorporate into this system.

While driving through a particularly remote stretch of TX highway today I stumbled across the following:

Intel S2600WT2 Server Board for Xeon Processor E5-2600 Family 24 DIMMs 10x SATA | eBay

Every box I've built to date has used a SM motherboard, so the Intel products and naming schemes are relatively unfamiliar to me. Based on the information I was able to gather via cellphone, it looks like this board (S2600WT2) will support at least 4x NVMe drives (possibly with an AOC/or the Intel NVME hotswap kit).

I noticed on Intels site that a "S2600WT Family" will support up to 8x NVMe devices:
Intel® Server Boards

My questions are these:
1. Other than the proprietary FF, why was this board so cheap... what did I miss?
2. Is it safe to assume that the "S2600WT2" is part of the "S2600WT" and will support up to 8x NVMe drives? If so, does this require that I use V4 processors?

Flame suit on for both buying while driving and lazily asking you guys to spoon feed me info


Thanks,
Craig
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,046
1,583
113
CA
It's not simply a "proprietary FF" it requires special power connections, etc... not your standard motherboard that drops into a SuperMicro or other generic style chassis or case and off you go.
 
  • Like
Reactions: Jeggs101

Jeggs101

Well-Known Member
Dec 29, 2010
1,489
228
63
Yep. Need Intel PSUs case backplanes and for nvme usually an upgrade kit.
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
Understood.. intent was to keep an eye out on eBay for a compatible bare Intel 2U chassis and build it out.
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
I suppose I'm comparing this to every other similar item listed and wondering why it's 50% BIN. Seller appears to be somewhat reputable..

Also looking at what an X10DRU-I+ goes for...
 
Last edited:

Patrick

Administrator
Staff member
Dec 21, 2010
11,971
4,932
113
The other suggestion I may have here is to look out for a cheap S2600WTTR or other WTT board. I think the second T means onboard dual 10Gbase-T instead of 1GbE.
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
One down, many to go:
Intel R2000WT server chassis RT2000 | eBay

Relative to what I'm used to with SM, Intel's documentation is flat out impressive.

Patrick, I had already purchased the board when I linked it. Relative to pricing from other vendors, I figured I'd have no problem getting rid of it @$200 if necessary.

Because I ended up with the Gnodal 72 port 10G switch (which is overkill for me) I was planning to buy several Chelsio T580-LP-CR or T580-CR 40GB NIC's and aggregate ports on the switch. After spending some time with all of the documentation from Gnodal, this would appear to very much be supported and should get me into a 40G network.
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
On that note, if anyone is looking for documentation on the Gnodal switches shoot me a PM and I'd be happy to provide what I've got. Due to the size, it will need to be via Dropbox or similar.
 
  • Like
Reactions: T_Minus

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
So from page 63 of the configuration guide, the trick to getting 8x2.5" NVMe setup appears to be the following option kit: A2U44X25NVMEDK2

The A2U44X25NVMEDK option kit (page 59) was 4xNVMe & 4 SAS3 ....reviewed here:
4 solutions tested: Add 2.5" SFF NVMe to your current system

Provantage has the "EDK2" variant NVMe kit @$519 which is the best price I found. Amazing.. drive cage and AOC's cost more than the case and MB combined!

Due to a few grammatical errors & seemingly ambiguous/conflicting statements on page 63, I'm unclear was to whether both the A2U44X25NVMEDK and A2U44X25NVMEDK2 kits are required to hit 8xNVMe or whether the A2U44X25NVMEDK2 might get the job done by itself populating all 8 trays with NVMe drives...

Can anyone who might have some first hand experience here chime in?

Configuration manual located here:
http://www.intel.com/content/dam/su...server-products/S2600WT_Config_Guide_2_10.pdf


Edit: Reading on it would appear that both kits may be required...?
 
Last edited:

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,046
1,583
113
CA
I'm pretty sure you'll need two kits.

As you can see it adds up fast :) I snagged an Intel NVME kit "Great Deal" many many months ago and just found the chassis it would work in for non-retail price just recently. I've yet to find a 2nd deal on NVME Kit, and would really like a 2nd ha ha :) for my other chassis.

And I agree with you, the Intel manuals and routing for cable, etc, is very nice and clean and easy to work with IMHO.

Also, while I know the E5-46xx work in supermicro 2P (E5-26xx) based boards can anyone confirm they would work in Intels own boards? I've heard they're a lot more picky.
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
I'm pretty sure you'll need two kits.

As you can see it adds up fast :) I snagged an Intel NVME kit "Great Deal" many many months ago and just found the chassis it would work in for non-retail price just recently. I've yet to find a 2nd deal on NVME Kit, and would really like a 2nd ha ha :) for my other chassis.

And I agree with you, the Intel manuals and routing for cable, etc, is very nice and clean and easy to work with IMHO.

Also, while I know the E5-46xx work in supermicro 2P (E5-26xx) based boards can anyone confirm they would work in Intels own boards? I've heard they're a lot more picky.
I should know here pretty quick..

I think I'm going to suck it up and buy they second kit. Sure I'll have $400 in the case and motherboard and $900 in the drive cages and cards, but overall that's not bad considering what Intel/Supermicro NVMe ready barebones kits are going for.
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
...now if I could just get @workingnonstop to reply to his FS thread so I can buy some of those 1.6TB SAS3 drives to fill up the other 8 slots, I'd be set.
 
Last edited:

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,046
1,583
113
CA
LOL yeah, I wish I had the 2U version not the 1U... only 4x NVME and 4x SAS3.
Better than none though :)
 

Rand__

Well-Known Member
Mar 6, 2014
4,592
912
113
Would this board support E5-26xx ES/QS CPUs if the E5-46xx happen not to run?
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
Would this board support E5-26xx ES/QS CPUs if the E5-46xx happen not to run?
Feel free to correct me if I'm wrong, but I've yet to see a report of the QS processors not running in place of the S-spec chips. ES is a different story.
 

Rand__

Well-Known Member
Mar 6, 2014
4,592
912
113
True enough, I just was afraid Intel's own might be especially picky maybe:) So lets say ES;)
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
So the plan is to go pick up the 4655v3 processors this afternoon, however I've been doing some research which has led to some second guessing..

The 4655v3 would seem to be more frequency optimized/oriented with 6C 12T (ea), a max single core turbo rating of 3.2GHz, and an all-core turbo frequency of 3.0GHz. This white paper from Samsung would suggest that NVMe storage performance (Ceph) was highly influenced by core count and overall processing throughput capability (Pg. 11, CPU Sizing):
http://www.samsung.com/semiconducto...orage_Using_Samsung_NVMe_SSDs-WP-20160622.pdf

This site has taken the time to provide throughput performance (
performance = "Effective core count" * All-core-turbo Clock (GHz) * Special ops.)
of various v3 processors (Table 2, midway down page):
Xeon E5v3 All Core Turbo and Amdahl's Law

It's my understanding that FreeNAS is optimized to utilize multi-core processing, however most FreeNAS hardware guides I've encountered to date have spec'd relatively low end processors (i3's, E3's, E52620's, etc). Not many of these guides have really been geared towards a build packed full of NVMe and SAS3 storage though...

Am I making a mistake by selecting a relatively low core count frequency optimized processor? Would I be better off with solutions offering a higher core count like the E52683V3?

In this scenario my backup plan would generally be to swap the 4655v3's with the 2696v4's from my other machine (22C/44T, FTurbo 3.7GHz, FAll-Core=2.8GHz), but unfortunately they would exceed the S2600WT2's TDP limit of 145W and, as an OEM variant of the 2699v4, may be blocked on an Intel board...
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,046
1,583
113
CA
@Rand__ yes Intel boards are much more picky than SuperMicro, and my understanding is it's rare for an ES CPU to work in an Intel branded motherboard.
I'm gonna throw some in myself (QS) and test this week likely. I think my motherboards are in the same family as these too, they look nearly identical.
 

Rand__

Well-Known Member
Mar 6, 2014
4,592
912
113
@AJXCR - FreeNas recommendations usually focus on the corporate utilization - lots of users -> many cores.
All peak performance recommendations per user recommend high frequency builds due to single thread issues with some of the relevant tools - namely SMB and maybe nfsv3 (afaik).
So depends on how you'd share out your data

@T_Minus - Sounds good:)
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
@AJXCR - FreeNas recommendations usually focus on the corporate utilization - lots of users -> many cores.
All peak performance recommendations per user recommend high frequency builds due to single thread issues with some of the relevant tools - namely SMB and maybe nfsv3 (afaik).
So depends on how you'd share out your data

@T_Minus - Sounds good:)
I intend to tie this box into two of my company's locations and will allow some level of access to certain key employees. I would imagine, however, that for now 10 users would be a worst case scenario. At the same time, for a relatively slight increase in upfront costs I could go with far more powerful CPU's..

No one would happen to have a set of 2743, 2667, or 2687W v3/v4 processors they'd be willing to part with? If trading out the 2696v4's I would prefer to do so with a higher spec frequency optimized processor...

The 4655's are so cheap that I'll probably just go ahead and pick them up..