need motherboard recommendation ...

BLinux

cat lover server enthusiast
Jul 7, 2016
2,528
975
113
artofserver.com
i need some recommendations for an older system (mainly because I want to get it used and cheap) that has very low power consumption while having at least 3 or more x8 PCI-E slots. the main use case here is for doing bulk firmware flashing of PCI-E cards (storage controllers, NICs, etc.). i currently have a spare C2758 system I am using for this purpose, which is nice since it only draws about 30-40W, but it only has 1 PCI-E slot.

i haven't found any low power SoC system boards that have 3 or more PCI-E slots... at least from what i've looked at so far.
 

pyro_

Active Member
Oct 4, 2013
742
162
43
You probably wont find many if any SOC with those specs what you might want to look at is getting a normal ATX board and install a T series celeron/pentium/I3 chip as those are the low power version of those chips
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,528
975
113
artofserver.com
You probably wont find many if any SOC with those specs what you might want to look at is getting a normal ATX board and install a T series celeron/pentium/I3 chip as those are the low power version of those chips
thanks...

ok.. so I found this combination for about $85:

Supermicro X9SCM-F (has 4x PCI-E x8 physical slots, 2 are x4 electrically which is fine for what i'm doing)
Core i3-3220T (35W TDP, typical power is 28W)

I need to confirm if that CPU will work in that board. the CPU isn't as low powered as the SoC systems, but this is a good starting point.

Can anyone recommend something better? (better==more pci-e slots and/or lower power)
 

K D

Well-Known Member
Dec 24, 2016
1,431
309
83
30041
Since this is not going to be an always on system, does absolute low power matter?
 

pyro_

Active Member
Oct 4, 2013
742
162
43
Yes they should work fine

some of the E3 1220L chips have lower power consumption however they are going to be a LOT more expensive. If you want more PCIe lanes then you are going to be looking at the E5 series chips
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,528
975
113
artofserver.com
Since this is not going to be an always on system, does absolute low power matter?
absolute lowest power, no. but as much as possible within the "cheap" price range I'm aiming for, yes. when in use, the system will be powered on for several hours at a time.

Yes they should work fine

some of the E3 1220L chips have lower power consumption however they are going to be a LOT more expensive. If you want more PCIe lanes then you are going to be looking at the E5 series chips
yeah, i saw the 1220Lv2, but that chip alone would more than double the $85 cost of the above proposed combo.

i did find one other combo: X8SIL-F + i5-650 for $60, but 1 less PCI-E slot. there are claims it idles at about 30W. but the extra PCI-E slot of the X9SCM-F will save me a lot more time in the long run; a lot of time is wasted in POST waiting for the system to being booting, so the more PCI-E slots I can use simultaneously will save me time.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,528
975
113
artofserver.com
well, this didn't turn out so well.... I got the X9SCM-F + i3-3220T combo, and although the system boots up fine, it turns out this board, and many similar ones have an issue with IvyBridge CPUs that don't allow 3 out of the 4 PCI-E slots to work. This completely defeats my purpose for getting it, but it did idle at around 45W which was pretty good, and some of that power was running the 5 fans+backplane in the 836 chassis, so it probably really does idle at around 35W.

supermicro is telling me i should try a E3-1220 (SandyBridge) to see if that allows the other PCI-E slots to work again, but I think I might try a E3-1220L (20W TDP) for $10 more. i'm hoping the E3-1220L will salvage this project or all of this will get listed in FS section soon.
 

Joel

Active Member
Jan 30, 2015
811
162
43
39
I wouldn't worry about buying L parts for low power consumption. Those are designed for cooling constrained environments, not lower idle. Think of it like a speed governor in a car. The idle consumption is about the same as regular parts.
 

abq

Active Member
May 23, 2015
424
72
28
well, this didn't turn out so well.... I got the X9SCM-F + i3-3220T combo, and although the system boots up fine, it turns out this board, and many similar ones have an issue with IvyBridge CPUs that don't allow 3 out of the 4 PCI-E slots to work. This completely defeats my purpose for getting it, but it did idle at around 45W which was pretty good, and some of that power was running the 5 fans+backplane in the 836 chassis, so it probably really does idle at around 35W.

supermicro is telling me i should try a E3-1220 (SandyBridge) to see if that allows the other PCI-E slots to work again, but I think I might try a E3-1220L (20W TDP) for $10 more. i'm hoping the E3-1220L will salvage this project or all of this will get listed in FS section soon.
Hi BLinux,
Sorry to hear your X9SCM-F board isn't working with i3-3220T CPU. I have exact same combo, but only used 1 slot. Talk about blind luck! ...I was thinking of upgrading to E3 V2 CPU and making a soft switch, but now you have me concerned with what will work. Did Supermicro tell you what Ivy Bridge CPUs would work for all 4 PCIE slots?
Best Regards :)
 

i386

Well-Known Member
Mar 18, 2016
2,037
536
113
31
Germany
i3 has 16/20 pcie lanes if I remeber correctly. You will need a cpu with 40pcie lanes to use all pcie slots.
 

abq

Active Member
May 23, 2015
424
72
28
i3 has 16/20 pcie lanes if I remeber correctly. You will need a cpu with 40pcie lanes to use all pcie slots.
Hi I386,
Thanks much for your help and pointing out the PCIE lane shortage. ...So I assume no PCIE bridge switch on this board. Are there any LGA 1155 Ivy Bridge CPUs that allow all 4 slots to work at full rated speed (8e,8e,4e,&4e)? I did google search, and 16 x PCIE 3.0 seems to be max for E3 V2 CPUs. ...Curious that Supermicro would offer 4 PCIE slots and claim the 4 slots are 8m/8e,8m/8e,8m/4e,&8m/4e, but with no way to actually utilize them. Maybe split into 4 x 4e lanes will be good enough for Blinux, but I have a pair of 4x10gb cards, and pair of 2x10gb cards to use for soft switch :( ...assume 4xpcie2.0 thru DMI channel

EDIT:
per Supermicro:
C204 chipset
2x (x8) PCI-E 3.0*** slots,
2x (x4) PCI-E 2.0 in x8 slots

Best Regards, ABQ :)
 
Last edited:

BLinux

cat lover server enthusiast
Jul 7, 2016
2,528
975
113
artofserver.com
Hi BLinux,
Sorry to hear your X9SCM-F board isn't working with i3-3220T CPU. I have exact same combo, but only used 1 slot. Talk about blind luck! ...I was thinking of upgrading to E3 V2 CPU and making a soft switch, but now you have me concerned with what will work. Did Supermicro tell you what Ivy Bridge CPUs would work for all 4 PCIE slots?
Best Regards :)
Thanks for the sympathies :) ... such is the world of these highly integrated PC architecture these days... in the software world, we learned long ago that tightly coupled systems tend to be quirky and complicated, so people design software systems to have well defined APIs and loosely coupled architectures. the hardware world seems to be going in the opposite direction with more and more being integrated into the CPU. but, i'll end my "old man" rant recalling the days when every PCI/ISA slot just worked as expected and you could populate every SIMM/DIMM slot without being concerned about number of ranks, speeds, low voltage, and speeds to determine how many ram modules will actually work and at what speeds.

Back on topic, I think there's an issue with the PCH C204 chipset here and it's interactions with the CPU. It looks like 1 of the 4 PCI-E slots goes through the C204, and that's the one that consistently works. The other 3 PCI-E slots are directly connected to the CPU, and that is where the problems are. Supermicro wasn't super specific in their response, but they mentioned that PCI-E 2.0 cards might not work in the PCI-E slots connected to an IvyBridge CPU, which in many cases are PCI-E 3.0 compliant. The i3-3220T I have is an even more quirky situation, as it is IvyBridge, but only supports PCI-E 2.0 (Intel® Core™ i3-3220T Processor (3M Cache, 2.80 GHz) Product Specifications), where I think the Supermicro BIOS 2.x is expecting things to work at PCI-E 3.0 speeds. My guess, though I cannot confirm as I don't have the CPU, is that if you used a proper Xeon IvyBridge, the PCI-E 3.0 slots will probably work with only PCI-E 3.0 compliant cards. With a SandyBridge Xeon CPU, all the slots should operate at PCI-E 2.0 speeds. When you read the specs on the C204 (Platform Controller Hub - Wikipedia), it does seem it was initially designed around SandyBridge architecture and support for IvyBridge features came later as an after thought and required fixed BIOS or perhaps not at all.

Anyway, i just ordered a E3-1220L to see if that might move this forward to a usable state for me; really want all 4 PCI-E slots working.

i3 has 16/20 pcie lanes if I remeber correctly. You will need a cpu with 40pcie lanes to use all pcie slots.
that's a good point, according to Intel ARK, it only supports PCI-E 2.0 and in these configurations:

a) 1x16,
b) 2x8,
c) 1x8 & 2x4

Strangely, Intel ARK says the i3-3220T doesn't support ECC, doesn't support VT-d, and doesn't support AES-NI. But the Supermicro BIOS on this board with i3-3220T allows me to "enable" those features! And it is booting with Unbuffered ECC UDIMMs...

I think using i3 on this board just complicates things. I hope with E3-12xx v1, what works or doesn't work becomes more straight forward.
 

pricklypunter

Well-Known Member
Nov 10, 2015
1,607
470
83
Canada
The board in my wee box upstairs is c204 based, with a 1245v2 and 32GB of Unbuffered DDR3 RAM installed. I have 3 PCI-e 2.0 slots and 1 PCI slot populated and working without trouble. I think most if not all your issues will resolve themselves when you drop a Xeon in there :)
 
  • Like
Reactions: BLinux

BLinux

cat lover server enthusiast
Jul 7, 2016
2,528
975
113
artofserver.com
The board in my wee box upstairs is c204 based, with a 1245v2 and 32GB of Unbuffered DDR3 RAM installed. I have 3 PCI-e 2.0 slots and 1 PCI slot populated and working without trouble. I think most if not all your issues will resolve themselves when you drop a Xeon in there :)
yeah, that's exactly what i'm thinking too...
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,528
975
113
artofserver.com
Ok, so I got a E3-1220L sandybridge CPU today and unfortunately, the situation is only slightly better. With the SandyBridge CPU, slot 4 and slot 7 work perfectly, but slot 5 and slot 6 do not work at all. Same cards that worked in slot 4 or 7, when I move them to slot 5 or 6 don't work; that's one card at a time. If I populate all 4 slots, the 2 cards in Slot 4 + 7 work and the 2 cards in slot 5 + 6 are no show.

I've sent out another message to supermicro tech support to see if they have any suggestions.

So, what I'm learning here is that finding a motherboard with an abundant number of working PCI-E slots (4?), and energy efficient isn't as simple as I had hoped.

So, with that said, does anyone have any suggestion of a motherboard + CPU combo that is energy efficient (idle at around 45W or less) with at least 4 working PCI-E slots? I'm starting to think this X9SCM-F isn't the best platform for this...
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,528
975
113
artofserver.com
ok, before i rip out the X9SCM-F + E3-1220L combo, I thought I'd try something the Supermicro tech suggested; trying to install other PCI-E cards in slot 5+6. Well, I have a marvell chipset SATA controller by HighPoint. HighPoint claims this card to be PCI-E 2.0 x4. However, when I put this card in slot 5/6/7 (supposedly the slots connected to the CPU's PCI-E lanes), it shows link capability of 5GT/s at x2 (so that's PCI-E 2.0 @ 2-lanes, right?) BUT it says link status is 5GT/s at 1x!! So, this is strange in many ways... why does it show x2 capability on a supposed x4 card? And, why is it only connecting at x1?

Same card in slot 7 (supposedly connected to PCH) shows it with link status of 5GT/s @ x2, which is at least the max reported link capability. So, something is still strange about the PCI-E slots connected to CPU. And maybe in general across the board or HighPoint's marketing materials mistaken this x2 card for a x4?

so, this is both good news and bads news i guess. it does seem the slot 5+6 PCI-E slots *do work*, but not necessarily with any PCI-E card that fits. i'm going to try a few other cards I have laying around and see which ones work, and at what speeds. the ones that I was trying earlier that didn't work in slot 5 + 6 were LSI HBA controllers.

As a side note, I disconnected 2 of the 3 fans and the system went from 45W@idle to 36W@idle. So it looks like the fans cost about 4.5W each, extrapolating from that, the system without any fans would probably idle around 31.5W. That's about the same as my C2758 atom system, if only the PCI-E slots didn't have such issues....
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,528
975
113
artofserver.com
my mind is blown... i have no idea what's going on anymore... short story is, as of 30 minute ago, everything, and I mean *everything* seems to be working as expected now.

here's the sequence of events up to this...

after last night's attempt to test the SATA controller coming up with weird x2 and x1 links, this morning i thought i would do some more testing with a different card. i had a Mellanox dual port ConnectX-2 card, which is PCI-E 2.0 x8 laying around so I started testing it in the X9SCM-F:

slot 7: card is present in BIOS, links at 5GT/s @ x8
slot 6: card is present in BIOS, links at 5GT/s @ x8
slot 5: card is present in BIOS, links at 5GT/s @ x4
slot 4: card is present in BIOS, links at 5GT/s @ x4

so, it seems everything works correctly! i then also tried out a Broadcom 1Gbps NIC card that is PCI-E 1.0 x1. A strange "event" happened during this test of the 1Gbps NIC... I plugged it into slot 7 and the system powered on by itself! Not sure what that means... but in any case, the NIC shows up in all slots (4-7) at 2.5GT/s @ x1 just as expected.

one last thing I wanted to try, was the LSI SAS2008 HBA cards i originally tested with, which are actually Dell H310 with LSI IT firmware. These cards occasionally have an issue with the SMBus pins B5/B6. So, i taped those pins over and thought I would try them out again. sure enough, they show as:

slot 7: card is present in BIOS, links at 5GT/s @ x8
slot 6: card is present in BIOS, links at 5GT/s @ x8
slot 5: card is present in BIOS, links at 5GT/s @ x4
slot 4: card is present in BIOS, links at 5GT/s @ x4

wow.. so at this point, i'm thinking maybe all this strangeness with the PCI-E slots was because of this SMBus pin B5/B6 all along! to validate this, I removed the tape on the pins B5/B6 and tried the card in each slot again. Remember, the last time I did this with this card without the tape on B5/B6, it would only show up in slot 4 and 7.... strangely, it shows up on all slots now, at the correct link widths just as above!

So, i grab another H310 card I had around, and tried it in all slots, and sure enough, it works in all slots at the correct link widths. Next, I decide, I'm going to populate all for PCI-E slots with H310 cards and see if maybe too many of them caused the initial issue; but no, all 4 cards show up exactly as expected and at the correct link widths.

i don't know what happened... but everything is working right now? i hope it never regresses because I just don't get it. i'm actually going to try that Marvell chipset SATA controller again later to see if it links at full speed instead of the x1 i saw yesterday. this has been one strange journey...
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,528
975
113
artofserver.com
well, apparently there's just no happy endings here....

so, although some how the X9SCM-F + E3-1220L combo and all it's PCI-E slots seems to work now, I just found out that sas2flsh.exe for DOS does not work on Supermicro X9 systems! I would either have to use UEFI or run sas2flash in Linux I think. UEFI scripting is not something I want to learn right now. I can do bash scripting in Linux easily, but the megarec tool appears only available for DOS. So, I would have to boot DOS to run megarec, then boot Linux to run sas2flash and transfer the files from the DOS partition to Linux.
 

Blazer

New Member
Jul 25, 2015
10
4
3
possible that ur card has damaged the slot, I do remember some cards were proprietary to certain boards and required taping, was a long time ago i had read a thread similar, and the board slot was damaged.