Dell 3-Node AMD DCS6005

Discussion in 'Processors and Motherboards' started by Xeppo, Jan 23, 2014.

  1. spyrule

    spyrule Active Member

    Joined:
    Oct 8, 2013
    Messages:
    443
    Likes Received:
    41
    Is it the lower sled ?
     
    #361
  2. epiphanyplx

    epiphanyplx New Member

    Joined:
    Jan 27, 2016
    Messages:
    7
    Likes Received:
    0
    It is one of the lower ones, but they are both lower now as I removed one of them after I shorted it out.
     
    #362
  3. spyrule

    spyrule Active Member

    Joined:
    Oct 8, 2013
    Messages:
    443
    Likes Received:
    41
    If its sled 2 (the sled underneath the other sled that has a system in it, I had the same problem. It was shutting down because the controller chip was over heating. I attached a 40x40x10mm fan to the heatsink, and it kept the temps 20' lower on the chipset, and it made all the difference.
     
    #363
  4. epiphanyplx

    epiphanyplx New Member

    Joined:
    Jan 27, 2016
    Messages:
    7
    Likes Received:
    0
    That sounds like a good idea, I think I will try it

    Was there a spot to power fan on the mobo? Did you just point the fan towards the back?
    I saw another user who had cut a hole out of the top to place an exhaust fan there.
    Since I only have two speeds currently I was thinking about doing that as well.

    Thanks for your help
     
    #364
  5. spyrule

    spyrule Active Member

    Joined:
    Oct 8, 2013
    Messages:
    443
    Likes Received:
    41
    No, you want to point the fan DOWN into the heatsink so that it blows air across the heatsink. The rest of the air pressure from the chassis fans will do the work to evacuate the warm air. There is a standard fan header (actually, I think there is two) that you can utilize off of the motherboard. Obviously, with a smaller 40x40mm fan, there is no chance it will draw too much voltage/current to damage the motherboard (unless it fails in some spectacular way).

    Now if you've removed the upper sled, and both are on the bottom, then yes, I'd personally replace the front fans with quieter 80mm fans, and then install one or better yet two 120mm fans near the rear that SUCK air out, this will improve the cross airflow across the cpu's, and yet keep the noise down. Now, if there is enough room, you may even get away with mounting a traditional HSF to each cpu. I don't know if the motherboards can support mounting a standard HSF though, so that might be more custom work then its worth... unless you go liquid cooling, but now your talking $$$, and you might as well sell the whole darn thing and buy a newer gen C6100 that runs cooler and quieter stock.
     
    #365
  6. spyrule

    spyrule Active Member

    Joined:
    Oct 8, 2013
    Messages:
    443
    Likes Received:
    41
    If you do the above, don't hesitate to mount an 80mm fan pointing down into the CPU heatsinks to improve their temps. It can't hurt.
     
    #366
  7. epiphanyplx

    epiphanyplx New Member

    Joined:
    Jan 27, 2016
    Messages:
    7
    Likes Received:
    0
    Spyrule,

    Thanks for all your advice. I like the idea of putting some fans on the heatsinks to blast them with fresh air. Would I just cut a square out of the case in this case?? Or do you think even just having the internal air blowing directly on them would be enough along with front fans blowing air out toward the back?


    Additionally, I am wanting to eventually replace the third sled as I had to shut down some of my VMs with only 2.
    I see this one: Dell PowerEdge C6100/C6105 AMD Cloud Server Node w/ 2x Heatsinks 12x 4GB 5300P

    That one is correct but comes with CPU and RAM that I hopefully don't need.

    Then there is this one: 3DNG0 Dell Poweredge C6105 Server Motherboard 03DNG0 NODE Socekt C32

    But it looks different, do you know if it would work?


    Thanks
     
    #367
  8. spyrule

    spyrule Active Member

    Joined:
    Oct 8, 2013
    Messages:
    443
    Likes Received:
    41
    Well, if your replacing your 3rd sled, then cutting the lid is pointless work (since it won't work with the 3rd sled in.

    Also, if you re-install a 3rd sled, you can no longer install fans pointing down into the cpu's.

    Both sleds you linked don't come with cpu's btw.

    If it was me, I'd honestly, just save your money, and buy a newer TY edition of the C6100's. something like this:

    Dell PowerEdge C6100 XS23-TY3 4 Node Server 12 Bay 8x Intel Xeon L5630 96GB RAM
     
    #368
  9. epiphanyplx

    epiphanyplx New Member

    Joined:
    Jan 27, 2016
    Messages:
    7
    Likes Received:
    0
    Good points. As I just bought a house a new server is going to have to wait. I think for the time being I will just try the fans blowing directly down on heatsinks as you recommended and then upgrade to new server at a later date.

    Thanks for all your help
     
    #369
  10. spyrule

    spyrule Active Member

    Joined:
    Oct 8, 2013
    Messages:
    443
    Likes Received:
    41
    the fan on the chipset is equally important. It gets hotter then anything else in the entire system.
     
    #370
  11. Azlaroc

    Azlaroc New Member

    Joined:
    Dec 17, 2016
    Messages:
    1
    Likes Received:
    0
    I also picked up one of these servers on ebay. It was sold to me as a C6105 but ESXi is reconizing it as a DCS6005. I was able to get up and running thanks to this forum. My question is when I set the 2 NICS for etherchannel on my 3750 I lose access to the BMC. I found this blurb about BMC. "NOTE: If the first integrated network interface controller (NIC 1) is used in an Ether Channel team or link aggregation team, the BMC management traffic will not function on PowerEdge x8xx systems. The NIC teaming option is supported only on PowerEdge x9xx and xx0x systems. For more information about network teaming, see the documentation for the network interface controller."

    I believe this server is a 9th or 10th generation server. I need to ether channel these NICs but I dont want to lose access to the BMC/IPMI. I was hoping someone out there ran into this.
     
    #371
  12. spyrule

    spyrule Active Member

    Joined:
    Oct 8, 2013
    Messages:
    443
    Likes Received:
    41
    Just in case anybody is still using/buying this older hardware, I just tackled reviving my server and making it STUPID QUIET with no heat problems!!!
    Catch is, you need to remove the 3rd top sled, it can only run as 2 nodes for this to really work.

    I'll post pics in the next day or two to show you what I did, if anybody is interested.
     
    #372
  13. nthu9280

    nthu9280 Well-Known Member

    Joined:
    Feb 3, 2016
    Messages:
    1,426
    Likes Received:
    363
    Would love to know. Last year, I bought a C6100 2-node unit with pic16 FCB. gave up on it and it has been shut down.

    Sent from my Nexus 6 using Tapatalk
     
    #373
  14. spyrule

    spyrule Active Member

    Joined:
    Oct 8, 2013
    Messages:
    443
    Likes Received:
    41
    So I was going to type a long post on how to do it, but decided to simply post a couple of youtube videos (much easier for me).

    First off: Sorry for the portrait version. My apologies.

    Here is the video explaining the setup:


    Here is the video of the final results and temperatures achieved:


    I'm currently uploading the what was done video, I'll post it when it is published.

    I'll attach a bunch of pictures that were taken to show the details in stills as well.
     

    Attached Files:

    #374
    Last edited: May 1, 2017
  15. tdale

    tdale New Member

    Joined:
    May 5, 2017
    Messages:
    2
    Likes Received:
    0
    Anytime i reboot the node, intel boot agent, or searches hardware on the system (centos iso when i can get it to semi load) the ipmi ip drops packets and i lose connectivity to the server. Anyone have that issue and know how to fix it?

    My system:
    Device Power Status : On
    Firmware Revision : 1.3
    AuxFirmware Revision : 1
    Firmware Build Time : Feb 22 2010 12:51:29

    Thanks,

    Tom
     
    #375
  16. GerryBass

    GerryBass New Member

    Joined:
    Jun 22, 2017
    Messages:
    1
    Likes Received:
    0
    Hi guys, ... Do you still use those systems ? I both 2 of them with 48GB RAM each nodes for a total of 288gb ram. Using ESXi 6.0 on it.

    Did you got problems since ?
     
    #376
  17. Serj-Spb

    Serj-Spb New Member

    Joined:
    Aug 22, 2017
    Messages:
    2
    Likes Received:
    1
    Hi everyone. I am wondering - where on the C6005 (DCS \ Tyan S8208) motherboard can I find a PS_ON atx pin connector? I've seen a post in the nearby thread about C6100 motherboard but not sure if the connector near the molex power is the same as the one described here C6100 xs23-ty3 Node workstation (pinouts etc)
     
    #377
  18. Serj-Spb

    Serj-Spb New Member

    Joined:
    Aug 22, 2017
    Messages:
    2
    Likes Received:
    1
    Things got clear. I've requested a support from the vendor and they sent me this wonderful pdf.
    I think it will be usefull for everyone in this thread.
    Funny thing I've made the same ATX adapter pinout 1 hour before receiving this doc, all is fineand working good.
     

    Attached Files:

    #378
    ashevchuk likes this.
  19. Jim Wilson

    Jim Wilson New Member

    Joined:
    Mar 8, 2018
    Messages:
    1
    Likes Received:
    2
    So this string sure saved my life. i have 5 of these chassis and lost 1/2 the bmc when we had to move data centers. i was able to decifer the string and put together some more condensed instructions. i wanted to share in case and thank spyrule for all the hard work and diligense. with copper sl1 fans i have been able to bring down the sr5650 down to 56 degrees Celsius running dual mellanox cards. we were running about 85 degrees until this bios repatch and the sl1 fans. here are my instructions. hope they help. thanks again


    create a bootable dos usb thumbdrive using the rufus program
    copy all files that are in the files for bootable usb
    and it should have ALL the tools you could need.
    Very important. do in order
    flash bios by running
    c:\bios\flash
    after reboot
    go to c:\soc
    run socflash bmcback.rom anything.rom
    the anything.rom can be anything
    next is to change the MAC address to what is supposed to be
    c:\
    cd\kcs
    launch idnic
    notice the 2 mac addresses. the bmc last digit will always be one higher. for example
    NIC 1 00:E0:81:D1:5C:F2
    NIC 2 00:E0:81:D1:5C:F3
    BMC 00:E0:81:D1:5C:F4
    you can verify by looking at the big yellow power cord on the montherboard. the sticker there is the bmc mac
    type KCSFLASH MAC 00:E0:81:D1:5C:F4
    This may not be needed. but if you have issues see below
    The one final step once you've rebooted, is to shutdown, and reset the BMC on the motherboard (do NOT need to reset the BIOS though).
     
    #379
    Datamax and spyrule like this.
  20. Datamax

    Datamax New Member

    Joined:
    Jul 25, 2017
    Messages:
    6
    Likes Received:
    2
    I have a lot of these, and they are now having worse and worse random crashes ! :O

    It's like the whole system just halts, sometimes coming back for a minute or two on rare occasion. Often BMC stops responding as well, sometimes it (rarely) works.
    I have replaced all hardware couple times now, and it made no difference at all, only keeping the drives & same usage on the system. In the past when crashed not even display output, but now we got display output and NIC remains up, but everything is 100% unresponsive.

    Running debian 8 with latest updates, nothing on logs etc.

    Couple of times i have catched kernel message BUG: soft lockup - CPU#1 stuck for 23s! [swapper/1:0]
    But only from 2 of crashy nodes, usually nothing. That came via SSH too so it was not fully halted yet. This is getting worse and worse by the month. All of them run coolish, SR5650 temps are high as usual, but i will try to cool them down more now.

    Many nodes i have added extra heatsinks to some of the chips, this seems to make no difference. "ear temps" average to abt 25C (depending on node 21-27C), and temp seems to make no difference since higher temp nodes have no issues.

    Only thing the same on these systems is that they are active system and not fully idling, but all of them have plenty of resources still free tho.

    Above Jim's instructions: Can you also post the bios & flash tool files as well?
    I'll try next just to reflash everything, just in the off chance it happens to work. totally out of ideas by now :/
     
    #380
Similar Threads: Dell 3-Node
Forum Title Date
Processors and Motherboards Dell 3-node C6105 Mar 18, 2015
Processors and Motherboards Dell 3-Node AMD C6005 BMC Firmware Feb 26, 2014
Processors and Motherboards Dell Xanadu C6100 XS23-SB Board Config Issues Jan 7, 2020
Processors and Motherboards Dell C6100 XS23-TY3 2U 4-Node (8 CPU) one node BMC works other 3 not so much. Oct 23, 2019
Processors and Motherboards Anyone still using Dell C6100? Sep 2, 2019

Share This Page