Taming the C6100

Patrick

Administrator
Staff member
Dec 21, 2010
12,368
5,497
113
One. I don't have redundant power supply and I don't thing the one power supply can take multiple cables.
Ah. Interesting note. Mine does the same when I only have one power supply plugged in or one power cable plugged in (making the second PSU un-powered.)
 

benso37

New Member
Jun 25, 2013
18
0
1
Ah. Interesting note. Mine does the same when I only have one power supply plugged in or one power cable plugged in (making the second PSU un-powered.)
Hmm...Can someone else confirm this? I'd imagine everyone initially having this issue since the servers ship with one PSU.

Thanks Patrick.
 

OrangesOfCourse

New Member
May 15, 2013
21
0
1
Help!!

I just took delivery of one of these monsters yesterday and upon starting all nodes, I realized that the amber/green lights keep flashing. Has anyone experienced this with their unit. All nodes go through post without any issues. I've contacted the ebay seller but any help on here will be greatly appreciated.

Thanks
Make sure the power supplie is in the top (PSU1) PDB. I had the same issue and it was because they shipped plugged into the bottom PDU. It kept giving me the error that the PSU1 had a fault and needs to be replaced.
 

benso37

New Member
Jun 25, 2013
18
0
1
Make sure the power supplie is in the top (PSU1) PDB. I had the same issue and it was because they shipped plugged into the bottom PDU. It kept giving me the error that the PSU1 had a fault and needs to be replaced.
Mine was also shipped with the PSU in the bottom slot. I will move it when I get home.

One other thing I noticed is that the server spins up like a jet when I first turn it on but then within a few seconds, it sounds like a 1U. I expected it to sound like a jet on idle. I can use my cell phone to measure the db level but it doesn't sound that loud.
 

OrangesOfCourse

New Member
May 15, 2013
21
0
1
Mine was also shipped with the PSU in the bottom slot. I will move it when I get home.

One other thing I noticed is that the server spins up like a jet when I first turn it on but then within a few seconds, it sounds like a 1U. I expected it to sound like a jet on idle. I can use my cell phone to measure the db level but it doesn't sound that loud.
It's suppose to do that. The FCB is designed to spin the fan at full power until it gets temp readings from a node.
 

benso37

New Member
Jun 25, 2013
18
0
1
Make sure the power supplie is in the top (PSU1) PDB. I had the same issue and it was because they shipped plugged into the bottom PDU. It kept giving me the error that the PSU1 had a fault and needs to be replaced.
That's exactly what the problem was. I moved the PSU up and the amber/green flashing lights stopped. Hopefully, this will help others.

Thanks for the help.
 

benso37

New Member
Jun 25, 2013
18
0
1
Well, I have 4 of these server (Super Micro branded) at work and they do scream, even after the initial jet sounding engine. This guy sitting on my kitchen island is very quite. Granted I haven't installed anything on it yet but I expected it to be very loud. At it's current db, it will do just fine in my basement.
 

idea

Member
May 19, 2011
86
5
8
What is everyone's opinion on changing the RAM from the common 12x2GB (24GB) to 4GB, 8GB, or 16GB DIMMs? The bigger the DIMM's capacity, the less DIMMs you have to install, and the less power required/heat generated.
 

dba

Moderator
Feb 20, 2012
1,478
183
63
San Francisco Bay Area, California, USA
The most common RAM configuration on eBay is six x 4GB DIMMS, 24GB per node. Since that leaves six empty slots and the 4GB sticks are relatively inexpensive, there isn't much impetus to switch to larger DIMMS until you need more than 48GB per node, after which 8GB DIMMS make sense for up to 96GB/node.

On the other hand, 16GB dimms are still expensive enough that you'll not see many of them in a previous-generation server like the c6100.

So, in my opinion, 4GB DIMMS are the price/performance leader while 8GB makes sense if you are building a dense VM server.

What is everyone's opinion on changing the RAM from the common 12x2GB (24GB) to 4GB, 8GB, or 16GB DIMMs? The bigger the DIMM's capacity, the less DIMMs you have to install, and the less power required/heat generated.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,368
5,497
113
As a reference, I am using a mix right now. In the colocation we are using 8GB DIMMs on the 4 web nodes. The way overbuilt pfsense nodes are one CPU and 3x 4GB. In the lab here, currently all of the nodes are using 4GB DIMMs. 16GB DIMMs are what we are using for testing newer generation servers though.
 

tiporaro

New Member
Jul 7, 2013
6
0
1
So, I've been following these threads for some time now and finally took the plunge - happily got my C6100 in yesterday. I was contemplating jumping into a fan mod as it can easily get loud enough to hear throughout the house from the basement, but was caught off-guard early on with the cpu temps I'm reading out over IPMI.

While I have booted them all on successfully, I currently have just a single node in a bottom slot wired up to 6 hdd bays, with a simple linux shell booted up and idling. At boot (and continuously thereafter) I see temps of ~46C on one core, and 60-64 on the second - longer idling doesn't seem to bring down the temps, save a degree or two on the second core. This is under no load on L5520's, with the stock fans all "humming" along at 6900 RPM and the lid on to keep air flow directed. Ambient is cool (65F, or 17-18C as reported by the ipmi ambient sensors), and the other MLB temps are reported as: 1 - 59C; 2 - 56C; 3 - 52C. I hear most people saying they see the L5520's idle closer to 40 (or less), so this has kind of taken me by surprise. Does anyone have any thoughts on what's going on, as I'd greatly appreciate it. Thanks!
 

PigLover

Moderator
Jan 26, 2011
3,075
1,380
113
The CPU idling at 46c isn't that surprising, but if you are seeing a difference of >20c between the two CPUs then you've probably got something blocking airflow through the node. Misplaced cables, missing air dam, something else odd. Or perhaps a poorly seated heatsink.

There is no good reason for that much difference in a single node. Something is definitely wrong.
 

tiporaro

New Member
Jul 7, 2013
6
0
1
Thanks for the exceptionally quick reply - you were in fact exactly right. A SAS breakout cable had inadvertently got tucked in the main open channel towards the back socket - what I get for working on it in my excitement late last night. A quick rerun of the cabling has them idling at 49 and 53, so at least no major discrepancy between the two now. However, its still a little higher than I was thinking they would be, especially sitting without a node tucked in above it. I'm sure the cabling from the M1015 I added impedes some airflow, but is that a normal idle range given the open bay? I'm concerned what it would be when I put an active node back above it.
 

PigLover

Moderator
Jan 26, 2011
3,075
1,380
113
49-53c idle is a bit high but is not unusual for 55xx/56xx series CPUs. As for airflow, you will probably get lower temps with the node above it plugged in. The C6100 is designed for the node to form a channel with air forced over the heatsinks. With the node above it missing you have the majority of the airflow blowing by in the open space where there is little/no cooling benefit achieved.
 

tiporaro

New Member
Jul 7, 2013
6
0
1
Makes sense, that was an alternative explanation I was considering but hadn't gotten around to testing yet, but nice to know that the temps aren't completely out of sorts. If I can pick your help for one other thing while you're around, reading through these threads I know you started with the Evercool fans to cut down the noise, but since then you noted they might not be adequate. I've heard the San Ace ones mentioned among others as well for alternatives. Given your experience, what would you do for quieting the C6100 down if you were starting from scratch today?
 

PigLover

Moderator
Jan 26, 2011
3,075
1,380
113
I don't have personal experience with the San Ace, but if I were starting today I'd probably prefer those over the Evercool.

Also, if starting today, I'd probably buy 4-pin fan "extensions" (short cables with a 4-pin maile/4-pin female on the ends) and then clip the end and solder it onto the Dell cable. In case that wasn't clear - (1) cut the Dell cable off of the original fan (2) cut off and discard the male-end of a 4-pin fan extension cable (3) solder the two cables (use small shrink-tube to cover the solder joints). The manufactured connector ends will be much more reliable than self-crimped connectors.

If you do this you end up with "standard" conectorized cables and can swap in fans to experience or correct for errors.
 

amaranto

New Member
Jul 18, 2013
2
0
0
I really appreciate your efforts in taming c6100. I am planning to replace the fans as well. I am not sure whether I could get the fans mentioned. But I would like to ask a few questions.

1) How much air flow is enough to prevent the machine from overheating? I live in a place with high temperature (~30C in summer) and humidity (>80%). Is 80CFM enough?
2) Is PWM a must when choosing the fans?
3) Also, if I add a display card at the end, do I need to buy fans with higher CFM?

Actually, I am planning to use delta FFB0812EHE-SP15. Just because it is the same manufacturer of the original fans. I feel more comfortable to use the same brand as I am not familiar with fans at all.

Thanks
 
Last edited:

RimBlock

Active Member
Sep 18, 2011
838
28
28
Singapore
Just as a FYI I am discussing with some suppliers of the Sanyo Denki 80 fans to see if I can get them to put on the 6 pin connectors themselves and make sure the cable is long enough before shipping. It will then hopefully just be a case of slotting them in for people.

Would be good if anyone can decent close up pictures of the I can pass on to the distributors.

I will update if it comes to anything.

Regards
RB
 

amaranto

New Member
Jul 18, 2013
2
0
0
Hi all,

Just for update.

I have bought

FFB0812EHE-SP15, 25RMB($4USD) x 4 = 100RMB ($16USD)
extension wire, 2RMB($0.33USD) x 2 = 4RMB($0.66USD)
shipment to Hong Kong = 35RMB ($5.7USD)
free to change to 5 pin (my C6100 should be a older model with 5pin)

They will arrive next week. I will report after testing them.

Thanks.
 

ascareg

New Member
Jul 10, 2013
13
2
3
Portland, OR
FFB0812EHE-SP15, 25RMB($4USD) x 4 = 100RMB ($16USD)
I purchased 4x of these FFB0812EHE-SP15 fans from this still active ebay auction: http://www.ebay.com/itm/300836055625

Pretty happy with them so far. Definitely slower and pushing less air than the originals, but I can stand to run the C6100 in my office without wearing earplugs. With the stock fans, I was measuring about 78dB on the back side of the unit. With these, I'm measuring about 58dD. This is with a single blade running ESXi, but basically idle, and measuring with an iPhone app. This 58dB mark comes with fans running ~3000RPM and CPUs at 43C and 46C.

Quiet as it now is, I still may move it to the garage, it'd be nice to have even more quiet in the office sometimes.

Also, if starting today, I'd probably buy 4-pin fan "extensions" (short cables with a 4-pin maile/4-pin female on the ends) and then clip the end and solder it onto the Dell cable. In case that wasn't clear - (1) cut the Dell cable off of the original fan (2) cut off and discard the male-end of a 4-pin fan extension cable (3) solder the two cables (use small shrink-tube to cover the solder joints). The manufactured connector ends will be much more reliable than self-crimped connectors.

If you do this you end up with "standard" conectorized cables and can swap in fans to experience or correct for errors.
I wasn't too worried about these fans not working, so I didn't buy fan extensions or new ends. But as their cables weren't long enough, I did have to clip and solder all the cables together more or less as mentioned here. End result is quite nice, though.

One suggestion for any newbies like me... getting the rubber fan mounts out of the original fans is not a simple matter of pulling. :) That breaks them. :) Use a small screw-driver or needle-nose pliers to push them out.


Thanks for all the info on this thread! Really helpful!