Heatsink for Tyan s7012 inside Supermicro SC826

[fossxplorer]

I'm going to replace an AMD MB + CPU in an SM SC826 with Tyan S7012GM4NR / S7012GM4NR-B and 2 x Intel Xeon 5670. I wonder whether the 2x heatsinks SuperMicro SNK-P0038P will keep the CPU's cool enough to prevent any throttling?
Otherwise i'm thinking of of 2x active HSF SUPERMICRO SNK-P0040AP4, but much more pricey.
I/O shield doesn't follow the MB so i need to find it (searched eBay without luck).
Wonder how much thermal paste i'd need?
Other suggestions are also appreciated.

Here is the list:
Board: Tyan S7012GM4NR-B
CPU: 2 x Intel Xeon 5670
RAM: some used, but known to be good
HSF: SNK-P0038P (will this passive HSF keep Xeon 5670 cool enough?)
Adapter:Supermicro CBL084L Front Panel Connector 6"
Thermal past: ARCTIC MX-4 (4g) (need more tubes? ) how much do i need?
I/O shield: dunno where to get them or how to find the part no.

The manual for Tyan S7012 at http://www.tyan.com/manuals/S7012_UG_v1.1_06212012.pdf says:
. The mounting hole pattern of the S7012 matches the SSI EEB specification. Before continuing with installation, confirm that your chassis supports an SSI EEB motherboard.

I can't find info about SSI EEB for the SM SC826 2U chassis. Can anyone confirm whether this board will fit in or not?




TIA

 

[britinpdx]

I've used the Tyan S7012 motherboards before, they are good robust boards.

Supermicro server chassis are very well engineered and by design are intended to work with passive cooling (at the expense of noise). Here's the link to the Supermicro Heatsink Matrix that covers the range of motherboards and fans.

The SNK-P0038P is a good heatsink and will work well, but as a passive cooler will rely upon the chassis to provide adequate airflow (meaning more noise if that is a concern).

The SNK-P0040AP4 is a great cooler but it is too big for a 2U chassis. My go-to 1366 2U active cooler is the Intel BXSTS100C, but if you want to stick with Supermicro there is a 2U 1366 active cooler, the SNK-P0036A4.

In terms of thermal paste, think less than more. I typically use the Arctic Silver 5 and apply a "shot" less than half of the size of my little finger nail.

Good catch on the front panel adapter, you will need one. However, the front panel "header block" on the S7012 is left rear, so the CBL-084L may be too short. I believe I used the CBL-0068L which is longer (it really depends on the length of the ribbon cable to the front panel switch assy)

I was never able to find an I/O shield for the S7012 from a google search, I suspect that contacting Tyan may be the only avenue.

In terms of being able to mount the S7012 in the SC826, I have been successful with it's brothers, the SC823 and SC825, so I'm confident that the board will indeed physically fit into the case. The only issues that you may run into depend upon the rev of the chassis. Older chassis revs have fixed motherboard posts, some of which may not line up with the motherboard mounting holes, and may indeed contact areas on the bottom of the motherboard and cause shorts. This can be remedied be either judicious amounts of electrical tape over the post (not my preferred solution as sharp contacts may still push through), or use physical mechanical means (drill etc) to the fixed post to remove it.

For missing post locations (the opposite problem) I use these 6mm tall M3 hex standoffs and attach them to the motherboard with an M3 screw prior to installing the motherboard. This provides support to the motherboard when pushing in connectors, memory, cards, cpu's etc to minimize motherboard flexing.

The only possible down side to the S7012 (and this is really being picky as I'm focused on minimizing noise) is that the BMC appears to have two different temp zones, one for the CPU (controlling CPU0FAN and CPU1FAN headers) and one for the chipset/PCIe cards (FRONT 1 FAN, FRONT 2 FAN, REAR 1 FAN). The minimum PWM duty cycle is 30%, so even at idle the chassis fans are pushing a lot of air. The 5520 chipset runs hot and needs appropriate cooling, so this really isn't a bad thing, just something to be aware of.

Lastly, the 8pin EPS12V connector for CPU0 (PW1) is located way back in the right corner of the motherboard. Depending on the cable length from the power distributor, you may need an extension cable. Here's the link to the Supermicro CBL-0062L, although there are plenty of other options available.

 

[fossxplorer]

Thanks a lot for your detailed reply.
Good to hear SM chassis are good. Noise is not a problem for me as servers are co-located at DCs.
I think i'll go with SNK-P0038P passive heatsink since you indicate S7012's PWM duty cycle is 30% at minimum. Currently, i have passive heatsinks inside the SC826 chassis with 2x Opteron 6376, seems to work fine without any heat issues.
Also good to know about Intel BXSTS100C in other builds if needed.
Thank you for pointing out the length of the front panel adapter, i'm gonna order CBL-0068L to bring with me.

Other items shopping list:
- electrical tape, but also drill in case it's the way to go for fixed posts on the chassis that might not align with the MB.
- couple of Supermicro CBL-0062L
- M3 hex standoffs and M3 screws


So this is invaluable info from your experience for my preparation before traveling over to my DC.
Hope to be 99% prepared now
Hope to be able to travel within the next month to fix the server and you will the first one to know how it went

So thanks once more!

I've used the Tyan S7012 motherboards before, they are good robust boards.

Supermicro server chassis are very well engineered and by design are intended to work with passive cooling (at the expense of noise). Here's the link to the Supermicro Heatsink Matrix that covers the range of motherboards and fans.

The SNK-P0038P is a good heatsink and will work well, but as a passive cooler will rely upon the chassis to provide adequate airflow (meaning more noise if that is a concern).

The SNK-P0040AP4 is a great cooler but it is too big for a 2U chassis. My go-to 1366 2U active cooler is the Intel BXSTS100C, but if you want to stick with Supermicro there is a 2U 1366 active cooler, the SNK-P0036A4.

In terms of thermal paste, think less than more. I typically use the Arctic Silver 5 and apply a "shot" less than half of the size of my little finger nail.

Good catch on the front panel adapter, you will need one. However, the front panel "header block" on the S7012 is left rear, so the CBL-084L may be too short. I believe I used the CBL-0068L which is longer (it really depends on the length of the ribbon cable to the front panel switch assy)

I was never able to find an I/O shield for the S7012 from a google search, I suspect that contacting Tyan may be the only avenue.

In terms of being able to mount the S7012 in the SC826, I have been successful with it's brothers, the SC823 and SC825, so I'm confident that the board will indeed physically fit into the case. The only issues that you may run into depend upon the rev of the chassis. Older chassis revs have fixed motherboard posts, some of which may not line up with the motherboard mounting holes, and may indeed contact areas on the bottom of the motherboard and cause shorts. This can be remedied be either judicious amounts of electrical tape over the post (not my preferred solution as sharp contacts may still push through), or use physical mechanical means (drill etc) to the fixed post to remove it.

For missing post locations (the opposite problem) I use these 6mm tall M3 hex standoffs and attach them to the motherboard with an M3 screw prior to installing the motherboard. This provides support to the motherboard when pushing in connectors, memory, cards, cpu's etc to minimize motherboard flexing.

The only possible down side to the S7012 (and this is really being picky as I'm focused on minimizing noise) is that the BMC appears to have two different temp zones, one for the CPU (controlling CPU0FAN and CPU1FAN headers) and one for the chipset/PCIe cards (FRONT 1 FAN, FRONT 2 FAN, REAR 1 FAN). The minimum PWM duty cycle is 30%, so even at idle the chassis fans are pushing a lot of air. The 5520 chipset runs hot and needs appropriate cooling, so this really isn't a bad thing, just something to be aware of.

Lastly, the 8pin EPS12V connector for CPU0 (PW1) is located way back in the right corner of the motherboard. Depending on the cable length from the power distributor, you may need an extension cable. Here's the link to the Supermicro CBL-0062L, although there are plenty of other options available.

 

[fossxplorer]

One question that came to my mind while ordering different parts:
i need to be able to test this board with CPU, RAM, HSF etc at home without the SM chassis prior to the travel.
Is there a clever way to do that by using on other cables with on/of switch?
Or can i use the CBL-0068L and shortcut the correct power pins?

 

[britinpdx]

You should be able to do this "on the bench" by connecting the appropriate power supply with 1x24pin main atx power cable, 2x8pin 12v power cables, and carefully shorting pins 11 (PWRSW+) and 13 (GND) on the J9 (TYFP1) front panel header block to turn on the power supply.

I'll check my notes and verify the CBL-0068L to header block connections that I used later in the day.

 

[britinpdx]

I think that these are the connections that I used for the S7012 front panel header block and the SM CBL-0068L ..

 

[fossxplorer]

That was very informative! Thanks again. I'll see if i can make use of it and get back here at a later time.

I think that these are the connections that I used for the S7012 front panel header block and the SM CBL-0068L ..

View attachment 2021

 

[fossxplorer]

@britinpdx,
finally able to test a bit. Installed Xeon X5650 in socket CPU0 with 3 DIMMs correctly populate, but after plugging mains, there is a LED going on, and afterwards starting to blink. Nothing happens when i press power on button on Sm CSE 826 chassis.
Is it that it can't boot with X5650 and need e.g an older E5520 chip? Or would hear beeps?
From TYAN - Download BIOS: TYAN (S7012):
*Note: If you are using Intel Westmere CPU’s, you must use bios v203 or higher. Any previous bios will cause system to NO POST when Westmere CPU’s are installed

Could it simply be the reason?

Here is a video with the blinking:
OwnDrive


EDIT: do both CPU socket need to be populated for the MB to boot at all?

 

[fossxplorer]

Ok, after going the hard route to get a Xeon L5520 today and installing it with no further progress, i found out that it was the power on button that's not working.
I was able to power it by shorting the pin 11 and 13. I was able to power on by shorting the pins directly on the MB and by the adapter cable as well.
There is a ribbon like cable going from the front panel of SM CSE-826 to the SM TB826 connector board inside the chassis (close to the chassis intrusion sensor). I unscrewed it from the chassis and there were exact same pins sticking out on the backside of the connector board.
So what i did was to connect the adapter to this connector and located the power pins, shorted the pins on the backside of it. It also worked, so that means the signal from the SM front panel when i press the red button isn't registered. Also, there is no other LEDs blinking, but i've only connected the HDD LED, and there is no HDDs.

Now i wonder, is there a way to fix this? I bought the chassis used on eBay.

Some images here: OwnDrive

 

[britinpdx]

It sounds from your troubleshooting that you have isolated the problem to the switch on the front panel. From memory the SC826 power button is in the left chassis handle. I've had to troubleshoot problems on an SC846 before that turned out to be connection issues on the thin flex cable that connects the buttons/LED's to the internal "breakout" pcb. (Not a well packed chassis when it was shipped to me).

In the meantime you can always use the IPMI capabilities of the S7012 to remotely start/stop the server. If I remember correctly the defaults are username = root, password = superuser

Is this a new issue for the chassis ?

 

[fossxplorer]

After thinking a bit over this at night, today morning i was able to connect from another chassis (the original chassis which is back from a DC due to other issues) the front panel into this chassis which i bought from eBay recently shipped from the US. And powering on from that chassis worked fine which confirmed my suspicion on the front panel. So i started to unscrew the left most 3 bays to get the plate that's holding the ribbon cable inside. I did this since i though i'd need to get the inner front panel adapter board disconnected from the ribbon cable. But when i took off the handle's part from each other, i could easily see the problem, see the images. The plastic "tightener" on the power button board was damaged. I was able to power on by holding the ribbon cable pressed down against the contact area now

Now i can order that part from SUPERMICRO POWER BUTTON BOARD PCB PN : FP826 E162061 + RIBBON CABLE, which seems to be the right one.
Images see here:
OwnDrive
OwnDrive

And the issue has probably been caused by bad packaging when shipping from the US to me, as i can clearly see the difference of the double package used by the DC and this simple packaging.


But yeah, your hint about powering on from IPMI is awesome, i was thinking all the ways to power on, but forgot this easy one! Yes, i was able to login in to IPMI with root and superuser, the console redirection worked (Windows Java),, but not from the Java in Linux (icedTea), which gave a strange XML issue about one of the XML tags not ending with a '/'.
But since the part is reasonably prices, i'd rather order a couple and fix it prior to testing further with the Tyan board, 2x Xeon X5670, some UDIMMs and shipping it back to the DC.

I might have done a mistake by not tightening the heatsink (SNK-P0038P) on the Xeon L5520 i'm testing with and the first run for about 45s to 1 min smelled a bit burned after turning off. I immediately tightened all of the 4 screws a bit further until i could feel there were "in place". Hope the smell was from the PSUs
But further boot has not been a problem. Just really hope i didn't damage the motherboard. The CPU i really don't care about, using it just for test and they are dirt cheap

Thanks for your GREAT help and your details!

 

[kdieiev2013]

Hi guys, does anybody had pinout info for Tyan s7012 power connector, both 24-pin and 8-pin, I wondering if only 12V power supply is enough
to power-up this board.
The second question is about TEMP_SENSOR pin on Front_Panel connector, is anybody attached temperature sensor to it, if yes what model it was? Linux lm-sensors reporting -156.0 degrees for that one. Manual states that it thermistor type, but what nominal?

 

[2wheelsdown]

Hi all,

Sorry to hijack - just created an account because I'm pretty desperate for some advice.

I have a tyan s7012 board which I'm using as part of a new home server build. It's being housed inside a nice roomy Phanteks Enthoo Pro case.

However, the two heatsinks I purchased, both Intel E97381-001's seem to have clearance issues with surface mounted components when I'm trying to mount them, fouling on things and generally just not fitting right.

So I guess, I'm looking for some decent alternatives. Size not really an issue because of the case, but they either have to secure to the 'inner' set of mounting screw holes, or if they do attatch to the 'outer' set of screw holes they need to be validated in some way that they definitely fit the board. Has anyone got any experience of fitting larger coolers to this board??

Oh, I guess I should mention, the CPU's are X5670's, so fairly 'hot'.

I was really gutted today when this happened. I was all set to build the machine and get some hardware tests underway, only to be hit with this.

Sorry if hijacking this thread isn't cool, let me know if so. TIA for any help offered.

 

[gr8ape]

I have used the BXSTS100C on that board without issue. But they are a little louder than you may like.


Sent from my iPhone using Tapatalk

 

[TheBloke]

I have the S7012 motherboard, and I also have 2 x X5670 CPUs. But mine are in a Chenbro 2U RM23212 chassis, with 3 x 5000 RPM case fans.

I just got the server a few weeks ago. When I got it, it had two slower CPUs installed - 2 x E5620, which are 80W 2.4ghz CPUs. I replaced this with the X5670s which are 95W and 2.93ghz, but I kept using the same heatsinks. That is because when I looked at proper heatsinks for the X5670, they were a) huge, and b) more expensive than what I paid for the CPUs themselves!

Below is my solution to improve the cooling on at least one of the CPUs. It's not pretty.. but it is definitely helping




Those extra two fans I added are both running at around 4000 RPM - they are fans I took out of an old server, the one I had before I bought this new (for me) Chenbro/Tyan.

They are connected to the Tyan motherboard's CPU 0 and CPU 1 fan connectors. The three case fans are on Front 1, Front 2 and Rear 1.

Checking my IPMI sensors, I always get "CPU0/1 below TMax" of no lower than 40-45° under full load.

Tyan's way of recording CPU temperature really confused me at first - it took me a little while to realise that with their way, higher is better! My idle temps are both CPUs at 65-66° below TMax.

I don't actually know what TMax is exactly. The Intel spec lists the TCase as 81.3°, but I can't have 66° below 81.3 as that would only be 15°! So I reckon Tyan have set TMax = 100° and so my 66° below is 34° at idle, which is great.

I don't know how applicable my solution will be to you in your case. On the left of my pics you can see the three case fans, which are normally running at 4000-4500 RPM. So the CPU on the left (CPU1 I think) is cooled almost entirely by this case fan. But the CPU on the right (CPU 0) was running 10 degrees hotter than CPU1 until I added its own fan directly against its heatsink, so I know my ghetto solution is having some effect there at least.

I added the second fan in-line with CPU1 as well, just in case it helped, but that one is more about helping to cool the RAM next to it. I don't remember if I checked CPU1's temps before and after adding that fan, but the temps are good overall so I'm happy.

The one area where I'm not happy is all my LSI cards. I regularly get "PCI Area" going above 40°, which causes all the case fans to go to 5500+ RPM. So I am planning another ghetto solution there - I am going to cut a 12 or 14cm square out of the case lid in that area, and fit a large fan immediately above the PCI-E cards. Right now during my testing I usually have the case lid open a bit and rest a fan over that area, and this keeps the IPMI reading for "PCI Area" at around 38-39° even when I am at full disk load. I have 4 x LSI cards connecting to 27 HDDs - 12 internal, 15 external.

 

[2wheelsdown]

Hey,

Thanks for the input. I'll update again with more info on my course of action shortly.

Btw, impressive HDD numbers there @TheBloke

 

[2wheelsdown]

So - I've ordered 2 x SUPERMICRO SNK-P0040AP4's.

They secure to the 'inner' set of screw holes on the CPU sockets I believe, so hopefully there should be no more clearance issues. [EDIT: Just looked at pics again, I may be wrong after all. F**KBEANS]

Not too worried about noise, as the system is stored in the garage. This also helps a lot with temps, as the ambient air temp is usually much lower than room temp. The case I have has a huge 200mm fan at front, and a 120/140mm fan at the rear too. There's room for 6 vibration damped HDD's plus 2 SSD's on the back of the MB tray to start with, and I'm sure I could squeeze a couple more in there in the future if the time comes...

@TheBloke How do you connect external disks to internal cards anyway?

@gr8ape I did look at the Intel cooler you suggested, but they were quite a bit more expensive so I went this way this time... Who knows, maybe I messed up and these won't fit either and I should've listened to you. EDIT: See other edit. lol

 

[TheBloke]

@TheBloke How do you connect external disks to internal cards anyway?

There's a good multi-part guide on this site, check it out:
External SAS/SATA Disk Chassis Wiring - Part 1

I have 12 disks in my Chenbro chassis, which are connected to 3 x 4-drive SAS/SATA backplanes, each having a single SFF-8087 port. So internally I have 3 x SFF-8087 cables from those backplanes to three ports on 2 x LSI 9201-8i cards, leaving a fourth port unused at the moment.

Then I have another 16 drive slots in an external JBOD chassis (occupied by 15 drives with 1 free slot). Well, I say 'JBOD chassis', it's an old server chassis with room for a motherboard and 16 drive bays. It was my old server (very old, like 2006-era AMD) which died a while ago, and I recently replaced with the Tyan/Chenbro. The old chassis now has has no motherboard, just the drive bays powered by a 600W desktop PSU.

Originally that server/JBOD chassis had 4 x 4-drive SATA backplanes presented as 4 x SATA2 ports on each backplane, ie 16 SATA2 cables required total. I recently took them out and replaced them with four more of the same Chenbro SAS/SATA backplane that came with my Chenbro chassis (namely the Chenbro 80H10321516A1 backplane, of which I now have 7 in total; three in the Chenbro and four in the JBOD.)

So those 16 slots in the JBOD chassis need 4 x SFF-8087 cables to access them. In the JBOD I have those 4 x 8087 cables connecting to 2 x 2-port SFF-8087 to SFF-8088 adapters, which are mounted in the JBOD (in the place where the motherboard's PCI slots used to mount). The use of those adapters is a key thing explained in that guide I linked above.

Then I have 4 x external SFF-8088 cables coming out of those adapters in the JBOD, to the main Chenbro server, where they simply plug into 2 x LSI 8e cards which have 2 x SFF-8088 ports each.

You can see the chunky 8088 cables in the first picture I posted - they're the really heavy duty black cables in the top right. Those are 2m cables which go off to the JBOD.

I've been working on this for the last 6 weeks or so. I started out simply planning to resurrect my old 9-drive ZFS array, getting it out of the old dead server and putting it in a consumer desktop PC case with a couple of fans... Instead I have fallen down the rabbit hole of upgrading everything - buying the proper Chenbro server, 27 new (used) SATA3 drives (selling my original, much slower 9 drives), 4 x LSI cards, all the cables and adapters I mentioned, plus the four backplanes in my old server/JBOD chassis.. and tomorrow I'm adding 10GBe networking gear as well! Because I figured what's the point of having all this space and performance if I can't properly use it from my workstation. Everything's been bought used and as cheap as humanly possible, but I've still had to sell pretty much everything I could find around the house that wasn't either essential or nailed down, to pay for it all.. Been fun though!

Once it's all done (which I really hope won't be much longer!) it's all going in my garage, like yours. Then I'll have a couple of 20m fibre cables from the house to the garage, for the 10GB networking.

Here's what it all looks like currently - I still have some cleaning up to do in the JBOD particularly, it's super messy and I haven't finalised the cooling in there, hence the multiple fans including ones outside the case! But that shouldn't take too much longer once everything is done.

Although my current problem is that I also bought a cheap PCI-E SSD for read/write caching, but after I add the 10GB network card I won't have any PCI-E slots free. So I'm now also looking into SAS expanders, so that I can hopefully run all 16 of my external drives from only one LSI card rather than two. To be honest my current config is a bit overkill in that I really don't need one LSI card per 8 drives when only using HDDs, so I'm hoping the expander will work and then I can sell at least one LSI to pay for it.

The upgrades are never ending..!