Intel S2600CP vs Supermicro X9DRi-iF

[Kapel]

First of all, as I am a new user - hello Dear Community!

I hope I'm posting in the right forum.

I would like to learn some hypervisors (KVM, ESXi, Hyper-V, Xen) and GPU Passthrough on them. To achieve this I would like to build a machine based on LGA 2011 v1/v2 (for further upgrade) and insert a few different GPUs (mostly PCI-E 3.0). In my idea, I would like to start in each VM and instance of GPU accelerated application just to prove it's working.

My struggle is between Intel S2600CP* and Supermicro X9DRi-iF because I can get those in approximately same price. I would like to insert 2x Xeon E5-2670 v1 and 64G of ram.

Here is my questions:
1) Which one is more stable with multiple GPUs?
2) Do both mobos accept a regular PSU or do I need to buy a dedicated one to any of those?
3) Do both mobos fit in a E-ATX (or any standard) case?
4) Should I consider any other motherboard? I'm aiming for a dual CPU.
5) Can I pick any CPU coolers or do they have to be dedicated to the mobo? I know about the LGA standard, but I'm afraid that two coolers might be too big next to each other.
6) What are the common problems with each platform?

All suggestions are welcome. I'm looking forward to build a 'best for the budget' rig.
ATM I've found an offer: 2x Xeon E5-2670 v1, Intel S2600CP, 64RAM for ~750 USD / ~720 EUR, I'm EU based.

*I do know that S2600CP has all x8 slots.


Thanks,
K.

 

[Ajgoyt]

K,

I will chime in a little i Have the same MB and will be using it for my Unraid server shortly...

1) Which one is more stable with multiple GPUs? - MY Supermicro seems real stable so far.

2) Do both mobos accept a regular PSU or do I need to buy a dedicated one to any of those? A regular PSU will work
3) Do both mobos fit in a E-ATX (or any standard) case? The supermicro looks like a normal ATX sized board
4) Should I consider any other motherboard? I'm aiming for a dual CPU. I will let the experts chime in on this one....
5) Can I pick any CPU coolers or do they have to be dedicated to the mobo? I know about the LGA standard, but I'm afraid that two coolers might be too big next to each other. YES you will need the correct CPU heatsinks for a LGA2011 slots. they sockets are close to each other on the supermicro.
6) What are the common problems with each platform? None for me so far - I will let some experts chime in.

All suggestions are welcome. I'm looking forward to build a 'best for the budget' rig.
ATM I've found an offer: 2x Xeon E5-2670 v1, Intel S2600CP, 64RAM for ~750 USD - Have you checked this website? NATEX - Refurbished Servers and Server Hardware they got great deals.....

Hope this answers most of your questions!
AJ

 

[Kapel]

K,

I will chime in a little i Have the same MB and will be using it for my Unraid server shortly...

1) Which one is more stable with multiple GPUs? - MY Supermicro seems real stable so far.

2) Do both mobos accept a regular PSU or do I need to buy a dedicated one to any of those? A regular PSU will work
3) Do both mobos fit in a E-ATX (or any standard) case? The supermicro looks like a normal ATX sized board
4) Should I consider any other motherboard? I'm aiming for a dual CPU. I will let the experts chime in on this one....
5) Can I pick any CPU coolers or do they have to be dedicated to the mobo? I know about the LGA standard, but I'm afraid that two coolers might be too big next to each other. YES you will need the correct CPU heatsinks for a LGA2011 slots. they sockets are close to each other on the supermicro.
6) What are the common problems with each platform? None for me so far - I will let some experts chime in.

All suggestions are welcome. I'm looking forward to build a 'best for the budget' rig.
ATM I've found an offer: 2x Xeon E5-2670 v1, Intel S2600CP, 64RAM for ~750 USD - Have you checked this website? NATEX - Refurbished Servers and Server Hardware they got great deals.....

Hope this answers most of your questions!
AJ

AJ,

Thank you very much for your fast response. I hope I'll gather as much info I can get to make the best decision!

I have another question regarding the heatsinks - do I need to buy the Supermicro heatsinks because of the dimensions (as a safe option) or would any other brand would do the job?

Oh, and I've edited my post adding the info about my localization. NATEX do have nice offers but it's cheaper for me to get it from somewhere from EU.

Cheers,
K.

 

[Ajgoyt]

No you dont have to buy Supermicro heatsinks/fans Noctua and lots of other vendors sell LGA2011 heatsinks and fans , you just need to make sure you read reviews etc as per some of these heatsinks can cover up memory/dimm slots ,if your running ram with big/tall heatsinks on them. see the picture of my board and you'll see how close the cpu sockets are.

hope this helps
AJ

 

[Ajgoyt]

I forgot to add this and its very important, the supermicro board does have what they call IPMI and a BMC controller and if you pick out fans that are quiter/spin slower than you will be setting off alarms in the BMC controller. which will ramp up the fans to compensate... see my recent articles of me trying to use quiter fans in my Supermicro CS846 - IPMI is pretty neat you can control the computer with it off, Security is an issue with IPMI and you can turn it off. The BMC controller i beleive will still run to protect the server from over heating if the fans speeds are not met. I found out how to tweek the settings for the BMC command line its.

 

[Kapel]

No you dont have to buy Supermicro heatsinks/fans Noctua and lots of other vendors sell LGA2011 heatsinks and fans , you just need to make sure you read reviews etc as per some of these heatsinks can cover up memory/dimm slots ,if your running ram with big/tall heatsinks on them. see the picture of my board and you'll see how close the cpu sockets are.

hope this helps
AJ

Thank you AJ!

This helps me a lot! I was afraid I would have to use the vendor's heatsinks or some server-like fans. Are there any problems with setting the RPM of the fans or does it adjust automatically? Sorry for the silly questions. For some reason I treat them as server boards and I'm afraid the fans would go full power.

My first choice for cooling would be a SilentiumPC Fortis. They're cheep, but good.

Is the Supermicro board very picky about the RAM?

Cheers,
K.

 

[Ajgoyt]

Thank you AJ!

This helps me a lot! I was afraid I would have to use the vendor's heatsinks or some server-like fans. Are there any problems with setting the RPM of the fans or does it adjust automatically? Sorry for the silly questions. For some reason I treat them as server boards and I'm afraid the fans would go full power.

My first choice for cooling would be a SilentiumPC Fortis. They're cheep, but good.

Is the Supermicro board very picky about the RAM?

Cheers,
K.

See the the comments i just posted above this message - and also my thread about Fan Noise ... Ready to switch to a 4U SuperMicro Case from a Norco - Need Advice.

 

[Ajgoyt]

I used (2) of these kits to fill all (8) dimms = 64gb of ram - Samsung 8Gb. M393B1K70DHO-CKO: 32Gb. Kit - Server Memory - Natex.us

 

[Ajgoyt]

this comment on my thread is about fan noise on my case - I beleive with this board you will have to get fans that match the parameters of the BMC or you could have fans ramping up and down alot, Now you could use seperate fan controllers but then your somewhat unprotected... if there is an issue... see these comments from a active member on this forum........

Supermicro FAQ 17907 says to use generic ipmitool, as described (for example) here.

In general, I don't recommend changing either fan hardware or thresholds - the manufacturers choose those for specific reasons. They expect these systems to be run in a datacenter with a good supply of cool air, not in homes / small offices. This is particularly true with fixed-configuration device like switches where the heat output is well-known. The fans and settings on a Supermicro board may be "overkill" for a small config, but on (for example) an SC836 with 16 drives and a fair number of expansion cards they're pretty close to optimal.

I know a lot of folks here are re-purposing equipment for home labs and I can understand that the gear is noisy, but do you really want to cook your expensive CPUs and other hardware by trying to make this equipment quieter? It might be a better idea to put the equipment in an out-of-the-way space (unless you live in a 1-room apartment) and extend the necessary ports (video/USB/etc.) with a commercial extender. Just my $0.02.

 

[i386]

Just my two cents:

• Supermicro comes with ipmi (mainboard with -f in the name) or without (mainboards without -f in the name), Intel mainboards support basic ipmi stuff but need extra hardware for ikvm, iso mounting and other cool things
  
• Intel mainboards ramp up the chassis/system fans to 100% if you don't use 6 pin intel fans (not a problem if you use it in a tower with consumer grade or low rpm fans)
  
• You can find more used Supermicro gear than Intel (at least that's my experience)
• Intel mainboards are designed for intel chassis, if you use other chassis you might need longer power cables and adapters.

 

[epicurean]

To add to what @i386 has said, SM boards are also much easier to passthrough more than 1 GPU in ESXi. I could never passthrough more than 1 GPU in my intel S2600IP4.
However if you get the Intel Boards in their P4000 casings, they are absolutely awesome in terms of build quality.

 

[Kapel]

Thank you for all the responses!

Before creating this post I was directed twoards Supermicro and wanted if it's the right path. I would like to apologize because I've mixed two different Supermicro motherboards.

The one I wanted (and hopefully everyone thought about) - X9DRI-F
a different one, which is quite ok for this project is the - X9DRD-iF

And here is another question about the X9DRI-F - according to @i386 comes with IPMI. The X9DRI-F unlike X9DRD-iF does not have a dedicated ipmi rj-45 port. I guess it's shared with one of the LAN ports, am I right? Does the motherboard need any mobo extension for the IPMI to work?

I know a lot of folks here are re-purposing equipment for home labs and I can understand that the gear is noisy, but do you really want to cook your expensive CPUs and other hardware by trying to make this equipment quieter? It might be a better idea to put the equipment in an out-of-the-way space (unless you live in a 1-room apartment) and extend the necessary ports (video/USB/etc.) with a commercial extender. Just my $0.02.

Yeah, but some of those people don't have their gear on 24/7. But I get your point!


Cheers,
K.

 

[Ajgoyt]

Thank you for all the responses!

Before creating this post I was directed twoards Supermicro and wanted if it's the right path. I would like to apologize because I've mixed two different Supermicro motherboards.

The one I wanted (and hopefully everyone thought about) - X9DRI-F
a different one, which is quite ok for this project is the - X9DRD-iF

And here is another question about the X9DRI-F - according to @i386 comes with IPMI. The X9DRI-F unlike X9DRD-iF does not have a dedicated ipmi rj-45 port. I guess it's shared with one of the LAN ports, am I right? Does the motherboard need any mobo extension for the IPMI to work?


Yeah, but some of those people don't have their gear on 24/7. But I get your point!


Cheers,
K.

If I understand your question about an extension - you must mean connection - see the picture its of the X9DRI-F IMPI is another RJ45 Jack off the back of the motherboard I/O

 

[Kapel]

So I've read something wrong again. Need to sleep more. Thanks again!


Cheers,
K.

 

[Ajgoyt]

Here is the link to all the info/ downloads etc about the board Supermicro | Products | Motherboards | Xeon® Boards | X9DRi-F

 

[Kapel]

Here is the link to all the info/ downloads etc about the board Supermicro | Products | Motherboards | Xeon[emoji768] Boards | X9DRi-F

As an irony I had that page open in a different tab. Well it's not the best start thread on a forum for me [emoji2]

For the record - some server makers share lan ports with ipmi. Lenovo does it on some servers. That's why I asked.

Best,
K.

 

[Ajgoyt]

I've heard that, keep in mind and sorry for bringing it up again IPMI can be a security risk, once I have my serve officially up and running I will need to secure it, right now the user and password are default Admin, Admin, but there are other tweaks that can be performed to keep the bad guys out. There are articles about IMPI best practices I've seen but haven't read..

AJ

Sent from my SM-G930V using Tapatalk

 

[Pradawny]

Just my two cents:

• Supermicro comes with ipmi (mainboard with -f in the name) or without (mainboards without -f in the name), Intel mainboards support basic ipmi stuff but need extra hardware for ikvm, iso mounting and other cool things
  
• Intel mainboards ramp up the chassis/system fans to 100% if you don't use 6 pin intel fans (not a problem if you use it in a tower with consumer grade or low rpm fans)
  
• You can find more used Supermicro gear than Intel (at least that's my experience)
• Intel mainboards are designed for intel chassis, if you use other chassis you might need longer power cables and adapters.

The 6 pin fans are not required for Intel s2600 motherboard. You just need to run updFRUSDR.nsh if you plan to use this board with non-intel chassis.

If the fans are very low speed, you propably will need to adjust some sensors parameters. You can find article with an example here -> LINK

 

[Ajgoyt]

interesting article about Intel boards and good find.

 

[Tdunbug]

In regard to the supermicro default low and High threshold triggers you will have to manually change them. This will stop the fans from racing up and down. There is a cmd prompt way to adjust the default settings of 450 and 1500. You can use IPMItool.exe to adjust the low and high settings. How To: change sensor thresholds with IPMI using ipmitool is a good tutorial.