Dell C6100 XS23-TY3 2U 4-Node (8 CPU) Cloud Server

PigLover

Moderator
Jan 26, 2011
2,976
1,283
113
The reason I re-asked the question is because of the following posts on page 32. PigLover - are you saying this because you tried or are the posts below not accurate?
Both first hand experience and an understanding of the architecture. The memory controller on the 55xx processors can't address 16gb per DIMM. While this poster might have gotten the 16gb DIMMs to work he probably forgot to look at see that he was only getting 8gb/DIMM available at the CPU.

Note that this same poster claimed elsewhere that he used DIMMs from both CPU banks with only one CPU installed...which is clearly FUD since the intel EP/QPI architecture makes this quite impossible.
 

c6100

Member
Oct 22, 2013
163
1
18
USA
Back to

"2. I am wanting to connect 6 drives to one sled's mobo - is my only purchase to make is Monoprice's 0.5m 30AWG Internal Mini SAS 36pin (SFF-8087) Male w/ Latch to SATA 7pin Female (x4) Forward Breakout Cable? Instead of the DELL CABLE SATA ASSEMBLY 6J3R2?
DELL CABLE SATA ASSEMBLY FOR DELL C6100 6J3R2"

I am not planning to plug into a RAID card but directly into the mobo. Therefore, the Monoprice option doesn't look like it will work. I also can't locate any of the Dell 6J3R2 cables. Anyone has any other option/source?
 

alan

New Member
Oct 24, 2013
20
0
0
is there a list somewhere of the available boards for the C6100 mezzanine slot? Am I correct that it normally ships empty?
 

Clownius

Member
Aug 5, 2013
85
0
6
Back to

"2. I am wanting to connect 6 drives to one sled's mobo - is my only purchase to make is Monoprice's 0.5m 30AWG Internal Mini SAS 36pin (SFF-8087) Male w/ Latch to SATA 7pin Female (x4) Forward Breakout Cable? Instead of the DELL CABLE SATA ASSEMBLY 6J3R2?
DELL CABLE SATA ASSEMBLY FOR DELL C6100 6J3R2"

I am not planning to plug into a RAID card but directly into the mobo. Therefore, the Monoprice option doesn't look like it will work. I also can't locate any of the Dell 6J3R2 cables. Anyone has any other option/source?

You need two sets of cable. From the board to the base of each node can be sata to sata or SFF 8087 to sata x4 (if your using a mez or Raid card). From there it has a something that looks like a pci type (obviously its not PCI but some custom thing) slot where it slides into the chassis. On the chassis side is a pair of SFF-8087 connectors. One does 1-4 and one does 5-6 (for some reason). Those then run via a cable to a backplane which has sata style connectors. So yes you will need the SFF-8087 to wire 6 drives to one tray. From the backplane to the bit the nodes slide into.
 

Clownius

Member
Aug 5, 2013
85
0
6
is there a list somewhere of the available boards for the C6100 mezzanine slot? Am I correct that it normally ships empty?
Depends who you buy it through. My last 24 drive version came with one per node. My first 12 drive version didnt.
 

dba

Moderator
Feb 20, 2012
1,478
181
63
San Francisco Bay Area, California, USA
is there a list somewhere of the available boards for the C6100 mezzanine slot? Am I correct that it normally ships empty?
I know of the following mezzanine cards for the c6100:

- dual-port QDR Infiniband with QSFP ports. Also does 10GbE
- dual-port 10GbE w/ SFP+ ports
- LSI SAS2008 six-channel HBA for internal drives. Uses a special cable with 1xSFF-8087 and 2x SATA connectors plus a management connector.
- LSI SAS1068E six-channel HBA for internal drives. Uses the same special cable.

You are correct that the used ones usually arrive with no cards, although you can get lucky sometimes.
 

Bumpy2020

New Member
Oct 5, 2013
6
0
0
Hi All,
I'm close to dropping cash on a C6100 with the L5639 CPU's (I need these for nested virtualization on ESXi 5.1+
The C6100 with L5520 CPU's came in with power consumption numbers (Taken from the first post)
Idle, 4x IPMI LAN, 8x Gigabit LAN conneted, 4x Intel 320 160GB, 2x WD Red 3TB: 174w

Questions:
1. Has anyone done this test with the L5639 version ?
2. Has anyone measure how many watts the chassis uses with 4 nodes connected to IPMI, but only one node powered on ? (Can power on other nodes via IPMI (vCenter DPM for example))

I know my old IBM x3850M2's this will replace draw a lot of power with just the power cable plugged in :(

Thanks in Advance
 

alan

New Member
Oct 24, 2013
20
0
0
I know of the following mezzanine cards for the c6100:

- dual-port QDR Infiniband with QSFP ports. Also does 10GbE
- dual-port 10GbE w/ SFP+ ports
- LSI SAS2008 six-channel HBA for internal drives. Uses a special cable with 1xSFF-8087 and 2x SATA connectors plus a management connector.
- LSI SAS1068E six-channel HBA for internal drives. Uses the same special cable.

You are correct that the used ones usually arrive with no cards, although you can get lucky sometimes.
I'm about ready to order my machine, I need to clarify one thing.

Am I able to put the Infiniband daughter card and a Lsi Logic Megaraid 9260-8i in the same node?
 

Madhelp

New Member
Feb 7, 2013
25
1
3
I'm about ready to order my machine, I need to clarify one thing.

Am I able to put the Infiniband daughter card and a Lsi Logic Megaraid 9260-8i in the same node?
Yes but beware that you might have to mod the node for the daughter cards ports to be accessed with the case closed.

Try jumper D from memory. I would test but im in AU and dealing with bushfires. Both my C6100's are in the car currently
Thanks again it worked like a charm.
 

alan

New Member
Oct 24, 2013
20
0
0
Yes but beware that you might have to mod the node for the daughter cards ports to be accessed with the case closed.
Can you elaborate?

I plan to get one of these daughter cards for each node:



and only one node will also have the Lsi Logic Megaraid 9260-8i

what do I have to modify?
 

JoelC707

New Member
Oct 25, 2013
4
0
0
35
Lanett, AL
Look at the two images below, see the section on each node just above the handle? In the top image the node has cutouts for the ports to stick through, the bottom one does not. Sorry the bottom image is blurry (and that both are so large), it's from ebay listings I've been watching.



 

JoelC707

New Member
Oct 25, 2013
4
0
0
35
Lanett, AL
Out of curiosity, does anyone know how long it takes to get one of these shipped from mobile_computer_pros? My company ordered a C6100 and C2100 from them (called them instead of via ebay) on Monday at the end of the day. I got the tracking numbers for UPS but they indicate they haven't been picked up yet.
 

alan

New Member
Oct 24, 2013
20
0
0
new question: Am I able to put the Infiniband daughter card and a Lsi Logic Megaraid 9260-16i in the same node?

 

Patrick

Administrator
Staff member
Dec 21, 2010
12,009
4,991
113
new question: Am I able to put the Infiniband daughter card and a Lsi Logic Megaraid 9260-16i in the same node?
No. The LSI 9260-16i will not fit the C6100 as it is a full height card. The C6100 only has a half-height card slot available.
 

alan

New Member
Oct 24, 2013
20
0
0
what's the best (reasonably cost) way to connect the 4 Infiniband daughter card's together? I've never used infiniband before.

I assume a switch and 8 cables, I don't know what kind of cable or switch to get.
 

c6100

Member
Oct 22, 2013
163
1
18
USA
You need two sets of cable. From the board to the base of each node can be sata to sata or SFF 8087 to sata x4 (if your using a mez or Raid card). From there it has a something that looks like a pci type (obviously its not PCI but some custom thing) slot where it slides into the chassis. On the chassis side is a pair of SFF-8087 connectors. One does 1-4 and one does 5-6 (for some reason). Those then run via a cable to a backplane which has sata style connectors. So yes you will need the SFF-8087 to wire 6 drives to one tray. From the backplane to the bit the nodes slide into.
Where do you recommend finding the parts?
 

Clownius

Member
Aug 5, 2013
85
0
6
I just used basic ones off ebay. A standard breakout cable works for both uses. Just try and find one the approximate length you need.
 

dba

Moderator
Feb 20, 2012
1,478
181
63
San Francisco Bay Area, California, USA
what's the best (reasonably cost) way to connect the 4 Infiniband daughter card's together? I've never used infiniband before.

I assume a switch and 8 cables, I don't know what kind of cable or switch to get.
You can connect three dual-port Infiniband nodes together into a loop, but for four or more nodes you do need an Infiniband switch. QDR switches are $1K or so, but DDR switches can often be found for a few hundred dollars. Properly configured, you can get 1,900MB/s out of DDR infiniband.
 

Bumpy2020

New Member
Oct 5, 2013
6
0
0
Hi All,
I'm close to dropping cash on a C6100 with the L5639 CPU's (I need these for nested virtualization on ESXi 5.1+
The C6100 with L5520 CPU's came in with power consumption numbers (Taken from the first post)
Idle, 4x IPMI LAN, 8x Gigabit LAN conneted, 4x Intel 320 160GB, 2x WD Red 3TB: 174w

Questions:
1. Has anyone done this test with the L5639 version ?
2. Has anyone measure how many watts the chassis uses with 4 nodes connected to IPMI, but only one node powered on ? (Can power on other nodes via IPMI (vCenter DPM for example))

I know my old IBM x3850M2's this will replace draw a lot of power with just the power cable plugged in :(

Thanks in Advance
Would anyone mind testing this for me ? Even on the L5520 as they are very similar TDP from memory (Just one of those plugin power meters)
My Home Lab needs an upgrade, as the x3850's are finally sold :) hehe