HP S6500 8 Node Chassis

bds1904

Active Member
Aug 30, 2013
271
76
28
Anyone have anything good/bad to say about the s6500 chassis? The prices are getting pretty good on ebay as of late, and you can get a sl390s G7 node w/ 10Gb network (or IB) for a great price.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,102
5,129
113
I have seen those. Prices are not bad at all. Storage is 4x 2.5" per node I think. The one thing I am not sure about is OOB management.
 

Patriot

Moderator
Apr 18, 2011
1,318
700
113
The SL390 is available in multiple configurations. 1u/2u/4u
1u typically has 2 LFF drives in the back but can have 4x SFF.
2u has a 4 SFF drive bay in the front and back has 3 slots for compute cards.
4u has an 8 SFF drive bay in the front and back has 8 slots for compute cards.
HP SL 390g7 quickspecs

The SL6500 enclosure can be mix and matched with g6/g7 and gen8 nodes but you can't mix and match full width with half width nodes.

Feel free to bother me with any questions on them.
 
  • Like
Reactions: Patrick

MiniKnight

Well-Known Member
Mar 30, 2012
3,001
911
113
NYC
I'm interested but the catch is that you have to want 8 nodes not 4. Slower to expand. @Patriot How's the ILO setup? Need to buy a key for kvm?
 

Patriot

Moderator
Apr 18, 2011
1,318
700
113
I'm interested but the catch is that you have to want 8 nodes not 4. Slower to expand. @Patriot How's the ILO setup? Need to buy a key for kvm?
ILO is the same for all. An Advanced key is needed to use kvm after the OS is up.
You could do 4/2u nodes or mix and match. Depends what you want to do with the cluster.

I am all Gen8 now...
I have 2x 270s, 2x 250s, 4x 230 for my 2 enclosures.
230=1u, 250=2u, 270=4u.

I also typically use a KVM direct attach rather than ILO.
 

jac

Member
Oct 21, 2012
48
0
6
Bellevue, WA
Having used both the iLO advanced option key on G7 HP's and the Lantronix Spider on many different system types I personally would spend the extra money on the advanced option key. The KVM spider resizes your open window on display resolution changes and makes it stupid painful to do things like jump into the BIOS on a reboot.

But if you're the frugal type and insist on the Lantronix, consider also getting the AC adapter so it's not power cycling to go along with the window resizing nonsense.

Mounting ISO's is also easier within the iLO. If I remember right, the Lantronix has to map to an SMB share.
 

tjk

Active Member
Mar 3, 2013
285
50
28
www.servercentral.com
I recently bought this exact unit:

HP Proliant S6500 8 x SE2170S Node 16 x X5650 512GB RAM No HDD Rails | eBay

Some early feedback:

The rail kit doesn't work in a Dell 4 post cabinet, not sure they will work in any 4 post cabinets. The unit with all 8 sleds is super heavy, you will need a special heavy duty shelf to hold it. No idea if these are original HP rails (they look like it), or after market junk rails, but they don't work either way, and have to spend another $150 bucks on a heavy duty 4 port shelf to support it.

There are 4 x 2.5" SATA HDD's options on each sled, however it requires a special HDD adapter to use them, and the vendor I bought it from doesn't sell them, nor can I find a vendor that sells them. I have some SSD's in them, so I sort of just plugged em in and let them sit there without being "locked in" to place. I wouldn't do this with spinning HDDs.

The PCIe slot is a nightmare to get to/use. You have to remove 4 screws (typical HP torq like crap), pull the cage out, then pull the pci riser unit out (2 more torq screws, different size), put your card in (I'm using dual port Mellanox QDR cards), put the riser and card back into the cage, and then screw the cage back in. It looks like 2 x SAS connections go from the motherboard to the riser, at least on the SE2170s sleds I have, which I've never seen used to extend the PCIe slot, very interesting.

There is no unit/chassis management module, at least that I can figure out how to access. Each sled has on-board IL100 with basic key, you'll need to provide an advanced key to get IPKVM functionality.

The firmware on them is quite old, and thanks to HP's new policy, you cannot just download new firmware for them without a maintenance contract. I "borrowed" a friends 09.14 SPP ISO, to see if it would update firmware, and after a few hours of running and scanning, it didn't find anything to update, not sure if the G6 sleds aren't supported or something else was wrong.

The IO is all out the front, so 3 x Ethernet (1 x ILO, 2 x network) and 2 x QSFP, which is a nightmare to manage and properly run and make look nice once you snake it from the front of the cabinet to the back and then to the TOR switches.

The fans are LOUD, you don't want to run this in a home lab, unless it is in another room/section of the house. Our units are in a datacenter so noise doesn't matter to us.

Summary, for my usage anyhow, great lab/play setup, good price per unit, however I'd spend a bit more and do Dell R610's if I was buying used and looking to put them into production.

Someone mentioned a good price for newer G7 sleds, can you link me? Anything useful on eBay is $2k, from what I can find.

Tom
 

bds1904

Active Member
Aug 30, 2013
271
76
28
HP Proliant SL390S G7 1U 2X Quad Core E5520 2 26GHz 8GB RAM No HDD | eBay
HP Proliant SL390S G7 1U 2X Six Core L5640 2 26GHz 8GB RAM No HDD | eBay
HP Proliant SL390S G7 1U 2X Six Core X5650 2 66GHz 8GB RAM No HDD | eBay

Pretty much pick what processor you want and buy some additional ram.

I never gave the HDD's sleds any thought as everything would be over a the 10Gb or IB. Boot would be USB. Heck you could even if I used a node as a OmniOS + Napp-It server with the use of an SAS HBA in the PCIe slot. If you are going crazy on the node setup you already have a rack to mount a 3U SAS expander in.
 

tjk

Active Member
Mar 3, 2013
285
50
28
www.servercentral.com
HP Proliant SL390S G7 1U 2X Quad Core E5520 2 26GHz 8GB RAM No HDD | eBay
HP Proliant SL390S G7 1U 2X Six Core L5640 2 26GHz 8GB RAM No HDD | eBay
HP Proliant SL390S G7 1U 2X Six Core X5650 2 66GHz 8GB RAM No HDD | eBay

Pretty much pick what processor you want and buy some additional ram.

I never gave the HDD's sleds any thought as everything would be over a the 10Gb or IB. Boot would be USB. Heck you could even if I used a node as a OmniOS + Napp-It server with the use of an SAS HBA in the PCIe slot. If you are going crazy on the node setup you already have a rack to mount a 3U SAS expander in.
Exactly, just need the single SSD for the OS (VMware or Hyper-V), all storage is run off dual Infiniband QDR ports, basically a cheap hypervisor cluster node.

What USB drive(s) do you recommend for OS loads that are reliable and will last?

Tom
 

DolphinsDan

Member
Sep 17, 2013
90
6
8
Don't get a USB for Hyper-V. ESXi = USB drive because Hyper-V has too many writes. Make sure you get the 16GB or 32GB USB thumb too. ESXi has failures with under 8GB and at today's prices, no point not getting the bigger drives.
 

Dajinn

Active Member
Jun 2, 2015
512
78
28
30
I recently bought this exact unit:

HP Proliant S6500 8 x SE2170S Node 16 x X5650 512GB RAM No HDD Rails | eBay

Some early feedback:

The rail kit doesn't work in a Dell 4 post cabinet, not sure they will work in any 4 post cabinets. The unit with all 8 sleds is super heavy, you will need a special heavy duty shelf to hold it. No idea if these are original HP rails (they look like it), or after market junk rails, but they don't work either way, and have to spend another $150 bucks on a heavy duty 4 port shelf to support it.

There are 4 x 2.5" SATA HDD's options on each sled, however it requires a special HDD adapter to use them, and the vendor I bought it from doesn't sell them, nor can I find a vendor that sells them. I have some SSD's in them, so I sort of just plugged em in and let them sit there without being "locked in" to place. I wouldn't do this with spinning HDDs.

The PCIe slot is a nightmare to get to/use. You have to remove 4 screws (typical HP torq like crap), pull the cage out, then pull the pci riser unit out (2 more torq screws, different size), put your card in (I'm using dual port Mellanox QDR cards), put the riser and card back into the cage, and then screw the cage back in. It looks like 2 x SAS connections go from the motherboard to the riser, at least on the SE2170s sleds I have, which I've never seen used to extend the PCIe slot, very interesting.

There is no unit/chassis management module, at least that I can figure out how to access. Each sled has on-board IL100 with basic key, you'll need to provide an advanced key to get IPKVM functionality.

The firmware on them is quite old, and thanks to HP's new policy, you cannot just download new firmware for them without a maintenance contract. I "borrowed" a friends 09.14 SPP ISO, to see if it would update firmware, and after a few hours of running and scanning, it didn't find anything to update, not sure if the G6 sleds aren't supported or something else was wrong.

The IO is all out the front, so 3 x Ethernet (1 x ILO, 2 x network) and 2 x QSFP, which is a nightmare to manage and properly run and make look nice once you snake it from the front of the cabinet to the back and then to the TOR switches.

The fans are LOUD, you don't want to run this in a home lab, unless it is in another room/section of the house. Our units are in a datacenter so noise doesn't matter to us.

Summary, for my usage anyhow, great lab/play setup, good price per unit, however I'd spend a bit more and do Dell R610's if I was buying used and looking to put them into production.

Tom
Sorry to necro the thread but would you still say this is worth it? A barebones S6500 all with SL390s nodes goes for 900 on the bay..that works out to 112.50 a "server", which, IMO, is pretty damn amazing before you add in CPU/RAM/Drives.

The attractive thing about a barebones with the SL390s is that they have 10Gb and IB onboard already...meaning I wouldn't have to shell out 80-100 for a card per node for my alternate choices. But does anyone know if that onboard IB is 40GBps?
 

Patriot

Moderator
Apr 18, 2011
1,318
700
113
Sorry to necro the thread but would you still say this is worth it? A barebones S6500 all with SL390s nodes goes for 900 on the bay..that works out to 112.50 a "server", which, IMO, is pretty damn amazing before you add in CPU/RAM/Drives.

The attractive thing about a barebones with the SL390s is that they have 10Gb and IB onboard already...meaning I wouldn't have to shell out 80-100 for a card per node for my alternate choices. But does anyone know if that onboard IB is 40GBps?
Edit... if it has the optional IB it is QSFP
HP ProLiant SL390s Generation 7 (G7) (QuickSpecs/c04123322.pdf)
 

Dajinn

Active Member
Jun 2, 2015
512
78
28
30
I can't get those fools to pick up the phone or reply to my ebay messages. ALso, the shipping for it is freight which could easily add 200 bucks to the cost. We'll see I guess.
 

tjk

Active Member
Mar 3, 2013
285
50
28
www.servercentral.com
I can't get those fools to pick up the phone or reply to my ebay messages. ALso, the shipping for it is freight which could easily add 200 bucks to the cost. We'll see I guess.
Mine shipped from TX to VA, and it was $175 bucks via freight. Those things are heavy, see my post above too, the rail kit I received was useless and I had to spent another $100+ bucks on a shelf to hold it.