Dell C6100 XS23-TY3 2U 4-Node (8 CPU) Cloud Server

[s0lid]

Well I picked one up yesterday and I am either hopelessly inept with it (it has been around 30 years since I last used one) or it is not working...

Setting to 20v direct and touching the probe ends together gives a reading of 0.00. I would have expected a number based on the 9v battery in the unit. Maybe it has a bad fuse .

If you get 0.00v on voltage range the multimeter is working just right. It's the ohm range you want to do pinout poking with or diode which will beep when there's 0ohm resistance.

 

[zoroyoshi]

Great thanks to RB!

c6100/c6145/c5220 mezzanine slot pinout (analyzing...)
side1,g,12V,12V,g,3.3V,3.3V,3.3V,3.3V,3.3Vaux,g,NSOp0,HSOn0,g,g,HSOp1,HSOn1,g,g,HSOp2,HSOn2,g,g,HSOp3,HSOn3,g,g,nc,key,g,HSOp4?,HSOn4?,g,g,HSOp5?,HSOn5?,g,g,NSOp6?,HSOn6?,g,g,HSOp7?,HSOn7?,g,g,refclk-,refclk+,g,RST,R13,R12
side2,g,12V,12V,g,3.3V,3.3V,3.3V,3.3V,g,nc,g,g,HSIp0,HSIn0,g,g,HSIp1,HSIn1,g,g,HSIp2,HSIn2,g,g,HSIp3,HSIn3,g,key,nc,g,g,HSIp4?,HSIn4?,g,g,HSIp5?,HSIn5?,g,g,HSIp6?,HSIn6?,g,g,HSIp7?,HSIn7?,g,g,R11,g,WaKe,R8

 

[c6100]

Hey guys I have some Dell Poweredge C6100 10 Gigabit SFP+ Dual Port NIC DPN TCK99 X53DF cards I am not going to use. Anyone interested?

 

[WScott66]

Hey all,

Sorry been rather busy at work (Surprising for being around the holidays...)
I got the unit and everything was good till I powered up the last unit and got sparks!
I worked with ESISO and the shipped me out a replacement node (Barebone w/heatsinks) the next day.
I presently have the unit up and running with 2 nodes running ESXi, 1 Win2012R2, and the last KVM.
I haven't taken any pictures of the unit as when I finally got chance to touch it I was heads down working on Certifications for work.

All of the Firmware was well out of date with the BMC at 1.04 as was the BIOS. I have them now upgraded to BMC: 1.33 and BIOS: 1.71. The FCB is version 16 which does not mat ch to the firmware available on Dell's site so it is at the revision received (No issues so far!). The unit did not come with 4 serialized nodes (Services tags being consecutive .1, .2, .3, .4 so they are definitely piecemeal the units together.) The unit came with all of the drive bays populated with sleds (All you have to do is add drives if you have screws.) which is a big plus as most sellers don't include all of the sleds. The Sleds for the nodes are the first generation so they do not have the slot for the Mezz card cutout.

ESXi5.1, 5.5, vCAC, vCOM, SRM... (on 60 day Partner Evals...), Hyper-V 2012/R2, etc... you name it its running in my lab!

Eg: With 24GB or ram per node I have 4 vESXi host running on the 2 physical ESXi hosts which represent the datacenters and SRM running across them with iSCSI w/SRA & VR in use for replication. and it all works as expected. I have nested VM's running Hyper-V W12/R2 and I couldn't be happier except I need more network bandwidth. so, that said, I am now looking to add 10GbE or 40GbI (QDR Infiniband) to the systems. Does anyone have contacts for these cheep?

PersonalJ, rnavarro, jschloer,

I saw your posts have you made the move and purchased your systems?
Loddy at ESISO says they have plenty... She can be reached at sma@esiso.com.
Make sure you tell her you were referred by me as I told her I would get the word out on their pricing.

 

[WScott66]

c6100,

What are you asking for the cards?
What quantity do you have?
Do you have the SFP modules? Fiber or Copper?

 

[RimBlock]

If you get 0.00v on voltage range the multimeter is working just right. It's the ohm range you want to do pinout poking with or diode which will beep when there's 0ohm resistance.

Yes, some kind soul pointed out me rather obvious mistake . I had a quick go and the connections on the back edge jump about on the PCIe connector so was waiting until I could take a decent pic and had some time to really give it some attention (i.e. the weekend).

Great thanks to RB!

c6100/c6145/c5220 mezzanine slot pinout (analyzing...)

side1,g,12V,12V,g,3.3V,3.3V,3.3V,3.3V,3.3Vaux,g,NSOp0,HSOn0,g,g,HSOp1,HSOn1,g,g,HSOp2,HSOn2,g,g,HSOp3,HSOn3,g,g,nc,key,g,HSOp4?,HSOn4?,g,g,HSOp5?,HSOn5?,g,g,NSOp6?,HSOn6?,g,g,HSOp7?,HSOn7?,g,g,refclk-,refclk+,g,RST,R13,R12

side2,g,12V,12V,g,3.3V,3.3V,3.3V,3.3V,g,nc,g,g,HSIp0,HSIn0,g,g,HSIp1,HSIn1,g,g,HSIp2,HSIn2,g,g,HSIp3,HSIn3,g,key,nc,g,g,HSIp4?,HSIn4?,g,g,HSIp5?,HSIn5?,g,g,HSIp6?,HSIn6?,g,g,HSIp7?,HSIn7?,g,g,R11,g,WaKe,R8

Although with zoroyoshis reporting of the pin out from the board (great work), is it still needed ?.

Maybe this should have its own thread to save it being burried in this monster thread.

RB

 

[s0lid]

Well the Pci-e datalines would need to be confirmed since they're marked with "?"

 

[RimBlock]

If you get 0.00v on voltage range the multimeter is working just right. It's the ohm range you want to do pinout poking with or diode which will beep when there's 0ohm resistance.

Great thanks to RB!

c6100/c6145/c5220 mezzanine slot pinout (analyzing...)
side1,g,12V,12V,g,3.3V,3.3V,3.3V,3.3V,3.3Vaux,g,NSOp0,HSOn0,g,g,HSOp1,HSOn1,g,g,HSOp2,HSOn2,g,g,HSOp3,HSOn3,g,g,nc,key,g,HSOp4?,HSOn4?,g,g,HSOp5?,HSOn5?,g,g,NSOp6?,HSOn6?,g,g,HSOp7?,HSOn7?,g,g,refclk-,refclk+,g,RST,R13,R12
side2,g,12V,12V,g,3.3V,3.3V,3.3V,3.3V,g,nc,g,g,HSIp0,HSIn0,g,g,HSIp1,HSIn1,g,g,HSIp2,HSIn2,g,g,HSIp3,HSIn3,g,key,nc,g,g,HSIp4?,HSIn4?,g,g,HSIp5?,HSIn5?,g,g,HSIp6?,HSIn6?,g,g,HSIp7?,HSIn7?,g,g,R11,g,WaKe,R8

Well the Pci-e datalines would need to be confirmed since they're marked with "?"

So that is not a charaterset translation issue .

Ok, so all I need to do is trace the contacts with the '?' to the standard PCIe slot pin on the riser then ?.

 

[zoroyoshi]

I marked ? (HSI4...7 and HSO4...7) because they are not connected to mezzanine slot in that riser card.
These signals from pci-e connector are connected like a loop-back (HSI4-HSO4, so on).
# not a characterset translation issue... but i'm sorry for my poor english.

 

[RimBlock]

I marked ? (HSI4...7 and HSO4...7) because they are not connected to mezzanine slot in that riser card.
These signals from pci-e connector are connected like a loop-back (HSI4-HSO4, so on).
# not a characterset translation issue... but i'm sorry for my poor english.

Don't be, it is fine. Thanks for contributing.

 

[WScott66]

All,

Does anyone know of a high-end display port graphics card that will work in the PCI slot on a node?
Looking to use one node as a workstation for CAD.

 

[chune]

All,

Does anyone know of a high-end display port graphics card that will work in the PCI slot on a node?
Looking to use one node as a workstation for CAD.

AMD 100-505634(100-505848) FirePro W7000 4GB 256-bit GDDR5 PCI Express 3.0 x16 CrossFire Supported Workstation Video Card - Newegg.com
AMD 100-505733(100-505846) FirePro V7900 SDI 2GB 256-bit GDDR5 PCI Express 2.1 x16 CrossFire Supported Workstation Video Card - Newegg.com
AMD 100-505746(100-505835) FirePro W600 2GB 128-bit GDDR5 PCI Express 3.0 x16 Workstation Video Card - Newegg.com

not sure what "high end" means to you, heres the search link:
Newegg.com - Computer Hardware, ATI, Workstation Graphics Cards, displayport

 

[RimBlock]

AMD 100-505634(100-505848) FirePro W7000 4GB 256-bit GDDR5 PCI Express 3.0 x16 CrossFire Supported Workstation Video Card - Newegg.com
AMD 100-505733(100-505846) FirePro V7900 SDI 2GB 256-bit GDDR5 PCI Express 2.1 x16 CrossFire Supported Workstation Video Card - Newegg.com
AMD 100-505746(100-505835) FirePro W600 2GB 128-bit GDDR5 PCI Express 3.0 x16 Workstation Video Card - Newegg.com

not sure what "high end" means to you, heres the search link:
Newegg.com - Computer Hardware, ATI, Workstation Graphics Cards, displayport

If you are intending on running the node in the C6100 chassis then you will need a low profile half length card. The V3900 seems to be the best that may fit that requirement although it is not really high end. From NVidia, a Quadro NVS 510 amy also fit. Matrox also do a few cards low profile, half length. I cannot comment on how good the cards are, this is mearly info on what may fit the chassis.

If you are looking to run outside of the C6100 chassis then anything really but you will need a PCIe extention cable or second riser to allow any non-half height, half length cards to clear the CPU socket and ram on that side. Just looking at placement of a full sized ATI 6970 on a C6100 node workstation build I am working on, the video card covers around half the MB, which is problematic for my watercooled design.

The PCIe slot in the C6100 is v2.1 so v3 cards will generally drop down to v2.1 speeds.

RB

 

[LawrenceGarvin]

I believe only the drive cage at the front is different. it is a removable part so if you can get a 2.5" one then you should be able to swap out the 3.5" one to turn a C6100 3.5" unit in to a 2.5" unit.
RB

In addition to the drive cage being different, so is the backplane, interposer, and the cable assemblies. But more importantly... which I painfully learned first hand, since I was actively trying to convert a 3.5" chassis to a 2.5" cage... despite the drive cage being a removable part... it's not, apparently, an exchangeable part. Dell Support was unable to provide me a part number, I've been unable to find a part number (or even a web page suggesting that such a part exists), and, in fact, in all 1100+ posts in this thread, not a single message discusses the availability of the drive cage as a separate part. In the end, I bought a barebones (no node) 2.5" chassis to meet my needs.

btw.. if anybody needs a 2.5" SAS backplane... it seems I now have an extra one.

Otherwise, I've read through all 1100+ messages of this thread (today), and through the first couple hundred on two previous occasions. My thanks to all of the geniuses who have contributed to this conversation. (I do wish I'd found it *before* I wandered off on this ill-fated convert-the-drive-cage project.) However, in the end, I'm going to end up with at least two systems: a 12-drive 2.5" system (which I plan to primarily equip as a WS2012 Scale-Out File Server) and an 8-drive 3.5" system which will primarily function as a Hyper-V v3 cluster. I've been very intrigued by the conversations about using SSD in place of HDD and I'm already looking into that for future expansion.

Only thing up in the air now is [a] steal 2 nodes from the existing 3.5" chassis to run the 2.5" chassis (which should be here next week), or move all 4 nodes from the 3.5" chassis into the 2.5" chassis and buy another 3.5" 4-node chassis and steal 2 nodes from it to create 2 2-node 3.5" chassis. I'm also very intriqued with the idea of putting 6 spindles on each node (in fact, getting to six spindles/node is why I wanted the 2.5" drive cage to start with.. <sigh>).

If anybody has any thoughts/suggestions.... I'm still in the planning stage. (In fact, my 3.5" drives finally arrived about an hour ago, so this weekend I finally get to fire up this 3.5" chassis -- if only to verify that it actually does before moving the nodes into the 2.5" chassis.)

 

[Chuckleb]

I must say that I am extremely impressed and delighted at how much the entire community has gathered around and hacked the crud out of these beasts. We rock.

 

[s0lid]

All,

Does anyone know of a high-end display port graphics card that will work in the PCI slot on a node?
Looking to use one node as a workstation for CAD.

I'm using 580GTX with pci-e riser. I suppose that counts as high-end ?

 

[brianmat]

I saw your posts have you made the move and purchased your systems?
Loddy at ESISO says they have plenty... She can be reached at sma@esiso.com.
Make sure you tell her you were referred by me as I told her I would get the word out on their pricing.

I just got a barebones quote from them and yeah, I think the prices are great. Could you PM me (or even post here) what you purchased and for how much? I wanted an empty server since I plan on going with some 6 core Xeons which we can move from 2 existing servers and we can migrate some memory around so it doesn't make too much sense for me buy 4 populated servers where I would just rip out some CPUs and RAM.

 

[c6100]

c6100,

What are you asking for the cards?
What quantity do you have?
Do you have the SFP modules? Fiber or Copper?

150 for the card, I have about 30

The are dual SFPs

 

[c6100]

I just got a barebones quote from them and yeah, I think the prices are great. Could you PM me (or even post here) what you purchased and for how much? I wanted an empty server since I plan on going with some 6 core Xeons which we can move from 2 existing servers and we can migrate some memory around so it doesn't make too much sense for me buy 4 populated servers where I would just rip out some CPUs and RAM.

Are you looking for a bare bones 3.5 or 2.5?

 

[WScott66]

I just got a barebones quote from them and yeah, I think the prices are great. Could you PM me (or even post here) what you purchased and for how much? I wanted an empty server since I plan on going with some 6 core Xeons which we can move from 2 existing servers and we can migrate some memory around so it doesn't make too much sense for me buy 4 populated servers where I would just rip out some CPUs and RAM.

I just got a barebones quote from them and yeah, I think the prices are great. Could you PM me (or even post here) what you purchased and for how much? I wanted an empty server since I plan on going with some 6 core Xeons which we can move from 2 existing servers and we can migrate some memory around so it doesn't make too much sense for me buy 4 populated servers where I would just rip out some CPUs and RAM.

I purchased a XS23-TY3 with the following configuration:
(1) 3.5" Chassis
(2) 1100W P/S & Cords
(1) Rails
(4) 250GB 3.5" SATA
(8) Empty 3.5" Sleds
(4) Nodes with:
(2) L5520 CPU's
(6) 4GB 1066 RDIMM's
For $650.00 (Including Shipping)
ESISO was great to work wit as I had an issue with one of the nodes which was shorting out against the chassis.
I sent them a picture depicting the issue and they shipped me out a replacement bare bone node the next day.
I have not been able to find anyone that would even come close to matching this deal! Closest was around 800 and Dell Outlet was around $8K for the same setup! (Ridiculous!!!!)