Dell C6100 XS23-TY3 2U 4-Node (8 CPU) Cloud Server

[nitrobass24]

I am sort of amazed they work. On Supermicro boards don't spicy chips normally lose support after the initial retail BIOS?

Yes usually, but since these are Dells, they typically do not remove the CPU codes from their BIOS's.
Also if there is ever Spicy Issues, I have found a quick call to SM to resolve the issue. They have no issue provided pre-release or inital release BIOS versions.

There are also some people that can update a current BIOS with the correct CPU code so you can still run the latest BIOS.

 

[Patrick]

I am sort of amazed they work. On Supermicro boards don't spicy chips normally lose support after the initial retail BIOS?

$125 each for 2. Bummed I didn't buy more (seller had 5.)

 

[Patrick]

Just to add on this. MrRackables is $700 with best offer for the barebones. Theirs has rails ($50), 4x drive caddy ($15/ea), but only 1 psu ($80 ea). Barebones with no rails, no caddy, but redundant psu... $225 on the forums.

I bought one of these as a cold-spare if the backplane or any of the PCB's fail in the first unit. Likely faster to just swap the entire thing out than to try opening up a machine to get it back in service. Also, if a motherboard fails, I'll have a spare populated with CPU/ memory available. Would make more sense if I had a second XS23-TY3 that was active, but at $225 (also ordered a bit more from them like drive trays), it made sense even now.

 

[Toddh]

This is really interesting. Looking at buying a couple for a Hyper-V 2012/Infiniband Cluster with DDS7 as HA storage. Anyone know how old these C6100s are?

 

[dba]

Patrick,

Do you (or anyone else) know the differences between the different C6100 versions? I've seen a number of photos of these servers and some have slide-out power supplies while others appear to have fixed power supplies.

For example:
http://regmedia.co.uk/2008/05/09/back_xs23.jpg
versus
http://en.community.dell.com/cfs-fi...00-11-metablogapi/6622.6220_5F00_634BBD7A.png

I just picked up a barebones unit from Visa Computer and was assured that it has the removable power supplies.

I bought one of these as a cold-spare if the backplane or any of the PCB's fail in the first unit. Likely faster to just swap the entire thing out than to try opening up a machine to get it back in service. Also, if a motherboard fails, I'll have a spare populated with CPU/ memory available. Would make more sense if I had a second XS23-TY3 that was active, but at $225 (also ordered a bit more from them like drive trays), it made sense even now.

 

[Patrick]

This is really interesting. Looking at buying a couple for a Hyper-V 2012/Infiniband Cluster with DDS7 as HA storage. Anyone know how old these C6100s are?

Todd - Very cool. Please keep us posted on the project's progress if you do. The cost per node on these is truly awesome.

dba - Here's what I know at this point:

• Dell C6100 XS23-SB - I think this was v1 - Seems to have sold decently well. Generally these came with the Core 2 generation Xeons, commonly the L5420's. Dual CPU. The L5420's are actually lower power consumption. For pure nodes, not a bad idea. For performance, the XS23-TY3 is the model to go for on the "inexpensive" side. Generally only save 100-200 on these and you have much lower memory headroom as they max out around 128GB, and even then memory costs go up a ton. Another note, you do not have the same upgrade path. Not positive if these are drive tray compatible but they did tend to have smaller power supplies.
• Dell C6100 XS23-TY3 - This is the version this thread is about, and I've bought (two of.) Generally come with dual L5520's. 6 RDIMMs/ CPU for 12 RDIMMs/ node/ 48DIMMs total. Thus far have successfully installed L5638's so it seems like you can upgrade to 32nm CPUs. These came in BOTH 2.5" and 3.5" varieties so that is worth noting. 95w TDP max CPUs only.
• Dell C6105 - This is the 4- node dual Opteron 4000 series version of the C6100. Lower volume product so much harder to find. If there were a bunch of used units on the market they would be ultra interesting given that they should be able to accept Opteron 4300 series CPUs.
• Dell C6145 - Only two nodes here, but of quad Opteron 6000 series chips so still 8 CPUs/ node. Not 100% sure but I think these only came in the 2.5" chassis. Looks like these support both Opteron 6100 and 6200 series CPUs so maybe Opteron 6300 also (not sure.)
• Dell C6220 - I look at this as rev2 for the C6100 series as noted above. Hard to find second hand. I think these go to higher TDP CPUs, have hot swap fans and etc. Xeon E5-2600 based nodes supporting higher than 95w TDP/ CPU configs and 1200w power supplies.

I have a spreadsheet I've been keeping the spare part numbers for the Dell XS23-TY3 that I was planning to update the first post with.

Hopefully that would help. Just off of the top of my head so consider directionally correct.

 

[Toddh]

Sweet, thanks for the info.

I found the infiniband card part# JR3P1.
http://www.ebay.com/sch/i.html?_trksid=p5197.m570.l1313&_nkw=JR3P1&_sacat=0&_from=R40

I asked VistaComputer if he had cpu & ram to go with the bare bones C6100 and he said they do. Thought I would find out how he compared to the $999 ebay price. Waiting to hear back from him on pricing.

 

[cactus]

The C6145 has a lot more available IO compared to similar density Supermicros (1U quad G34). Really amazing what Dell fit into 2U. The C410x is a vary interesting chassis also, obviously this line was designed for high density HPC.

 

[Patrick]

Been keeping a list of spare parts for the Dell C6100. I will keep a list of the main spares.

Dell C6100 XS23-TY3 Spare Parts List

Rackmount Rail Kit - 0Y3DX1
Mid-plane Chassis Fan - 104TT
1.1kw PSU - XVKF0
Drive Sled - 8TV68
Motherboard Tray - D61XP
Mellanox dual port QDR Infiniband Daughter Card - JR3P1
Intel 82599 10GbE Daughter Card - TCK99

 

[Patrick]

Patrick,

Do you (or anyone else) know the differences between the different C6100 versions? I've seen a number of photos of these servers and some have slide-out power supplies while others appear to have fixed power supplies.

Did you get yours yet?

 

[dba]

Mine is scheduled to arrive this Thursday.

Did you get yours yet?

 

[Toddh]

Patrick,

What do you think of your C6100 so far? Are you doing bios and/or firmware upgrades and anything unusual to report? I checked Dells website and there are a lot of recent drivers and firmware, 2012 - 2013, which was good to see so they are keeping up to date on this model. Didn't see any Windows 2012 drivers so I may have to do a little searching there. What OS are you loading?

The I ordered 2 from eBay Monday, should be here in the next couple days.

 

[Patrick]

So I have two at home right now. They are getting shipped in the morning. Everything seems to work well on them. One big note is that neither came with power cables. So I ordered four on Amazon.

As far as OS, using FreeBSD on the pfsense nodes and Proxmox VE for the "compute" nodes. I believe Proxmox 2.2 is Debian based. Sadly no Windows Server 2012 at the moment.

Now the big question is whether I buy a fourth one for home use and to play with the cluster stuff. Out of CPUs/ memory so debating this a bit. That would likely use Windows Server 2012. Would love it if you posted your thoughts. I think dba is getting one Thursday (see above) so would be great if we can all get some notes down for others.

 

[PigLover]

Anybody with any leads on the 10Gbe mezzanine card for the C6100? I cant identify the right part #.

 

[Patrick]

Anybody with any leads on the 10Gbe mezzanine card for the C6100? I cant identify the right part #.

Linked: Dell TCK99 10GbE Daughtercard

 

[Toddh]

Contacted eBay seller about when they would arrive here. He said they assemble each server from barebones which takes several days so mine have not been shipped yet. It does say on the eBay listing 3 - 4 days for processing, which I missed. Mine should come in first part of next week.

 

[Toddh]

If anyone needs ram let me know. I have a contact that has ram he assures me will work in the C6100. I believe its pulled from Dell R710.
48Gb = 6 x 8GB dimms $250
48Gb = 12 x 4Gb dimms $200

It was some of the better pricing I had seen on ebay so I saved his contact info.

 

[gigatexal]

is there a place for a disk to install an OS for each of the motherboards? these specialised builds are fascinating but not very intuitive

 

[PigLover]

is there a place for a disk to install an OS for each of the motherboards? these specialised builds are fascinating but not very intuitive

Your question could be interpreted two ways. A disk (hard disk) to install the OS to? Or a disk (CD or otherwise) to install OS from?

First version probably isn't what you were asking, but each server (MB) has 3 (3.5" model) or 6 (2.5" model) disk slots on the front dedicated to it.

Second way to interpret your question is more interesting. Is there a removable (CD or otherwise) to install OS from? Yes - its on your PC/LapTop/etc that you manage them from and you remote-attach it via IPMI. There are also USB slots on the back if you prefer using a USB install or portable CD.

 

[Patrick]

If anyone needs ram let me know. I have a contact that has ram he assures me will work in the C6100. I believe its pulled from Dell R710.
48Gb = 6 x 8GB dimms $250
48Gb = 12 x 4Gb dimms $200

It was some of the better pricing I had seen on ebay so I saved his contact info.

Just with this remember that the 5500/ 5600 series Xeons are going to be very sensitive to the number of DIMMs populated per channel. Not a huge deal for most applications but if you fill the DIMM slots with 1066MHz quad rank memory you end up running at 800MHz DDR3 max on the 5500 series as an example. In many cases more memory is going to be better than faster but it does depend on what you are running.

Todd, who did you buy from? I had the 3-4 day process time on my first one also. I ordered that one from pnieman. Also when I was at UnixSurplus/ MrRackables I got the impression that the CPUs/ memory were pulled from the units also so they would need to be configured.