HP SL6500 4U 8-Node (16 CPU) Cloud Servers | 256 Threads | 2,048 GB RAM ~$2000

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Zankza

Active Member
Feb 1, 2014
79
15
28
I've begun the "Quick Crash" post consisting of information to get you quickly online with those nodes. They are located at original post. Big thanks to those who have already shared information regarding this product.
 

raileon

Active Member
Jun 22, 2016
150
38
28
Should the ethernet routine doesn't go well for any reason, You may use SUV (Serial+USB+Video) to connect and start a terminal, And press "ESC+Shift+9" to get ILo login prompt and from there you may control (and execute same command set).
For this part I actually used a regular RJ45 -> DB9 serial cable. I don't have the SUV cable yet but I'm sure that works as well.

I ordered that cable yesterday when I went over to the server with a USB stick and realized... there is no external USB port lol
 

Fredrik

New Member
Mar 28, 2016
5
1
3
I ordered that cable yesterday when I went over to the server with a USB stick and realized... there is no external USB port lol
Fortunately, a workaround is available. If you have a USB extension/hub, there's an internal USB port which can be temporarily routed out for OS install.
 
  • Like
Reactions: raileon

raileon

Active Member
Jun 22, 2016
150
38
28
I played around with them some more today.

The integrated LSI 2308s are at the highest version available from HP which is 15.10.09 so SPP must've updated them. Avago/LSI have firmware for their 2308s at P20. No clue if you can even compare the version numbers directly and since these are so integrated into other parts of iLO and HP stuff trying to cross-flash might be a terrible idea. I'm going to leave them alone unless I run into any compatibility problems.

SPP definitely didn't update the chassis firmware which was 5.2 0r 5.3 (I forget which). So I updated that to the current 6.2 from within linux. Not sure what I gained here, sure as hell didn't help with sound, but it didn't break anything either.

The FlexLOM 544FLR QSFP cards are working well. I had to install the Mellanox OFED driver package first from the Mellanox site. This package will usually update your firmware too if you're using a regular card. In this case you have to grab the HP firmware and then burn it with mstflint. After a cold reset, my switch could connect to the card at full 56G on both IB and ETH so these seem to be identical to regular ConnectX-3 VPI cards.

I need to pick-up a bunch of 3.5" drive screws for all the PCIe/FlexLOM cards. This was posted in the deals forum: Supermicro HDD Screws (4.98$ for 100 screws w/ shipping)

PC Connection has both the 3.5 and 2.5 drive screws for $1.91 per bag but the shipping is terrible so it only works better than ebay if you order a lot.

Racks come in tomorrow. Hopefully I'll only need a screwdriver to get them mounted and not a dremel/hacksaw/hammer.
 

Zankza

Active Member
Feb 1, 2014
79
15
28
I played around with them some more today.

The integrated LSI 2308s are at the highest version available from HP which is 15.10.09 so SPP must've updated them. Avago/LSI have firmware for their 2308s at P20. No clue if you can even compare the version numbers directly and since these are so integrated into other parts of iLO and HP stuff trying to cross-flash might be a terrible idea. I'm going to leave them alone unless I run into any compatibility problems.

SPP definitely didn't update the chassis firmware which was 5.2 0r 5.3 (I forget which). So I updated that to the current 6.2 from within linux. Not sure what I gained here, sure as hell didn't help with sound, but it didn't break anything either.

The FlexLOM 544FLR QSFP cards are working well. I had to install the Mellanox OFED driver package first from the Mellanox site. This package will usually update your firmware too if you're using a regular card. In this case you have to grab the HP firmware and then burn it with mstflint. After a cold reset, my switch could connect to the card at full 56G on both IB and ETH so these seem to be identical to regular ConnectX-3 VPI cards.

I need to pick-up a bunch of 3.5" drive screws for all the PCIe/FlexLOM cards. This was posted in the deals forum: Supermicro HDD Screws (4.98$ for 100 screws w/ shipping)

PC Connection has both the 3.5 and 2.5 drive screws for $1.91 per bag but the shipping is terrible so it only works better than ebay if you order a lot.

Racks come in tomorrow. Hopefully I'll only need a screwdriver to get them mounted and not a dremel/hacksaw/hammer.
Would you mind help me understand where are you getting SPP? And if it's possible you can share them too?
 

Fredrik

New Member
Mar 28, 2016
5
1
3
The integrated LSI 2308s are at the highest version available from HP which is 15.10.09 so SPP must've updated them. Avago/LSI have firmware for their 2308s at P20. No clue if you can even compare the version numbers directly and since these are so integrated into other parts of iLO and HP stuff trying to cross-flash might be a terrible idea. I'm going to leave them alone unless I run into any compatibility problems.
JFYI, cross flash to P20 DOES work - use 9205-8e FW, as these cards do not support PCIe 3.0. However, be warned that the fan speed will increase to 70%. This happens with the latest HP P15 firmware as well, when using non-HP drives. I couldn't find a way around this, so I reverted back to the P11 firmware shipped with the system. Fan speeds back to normal. Oh well...
 

raileon

Active Member
Jun 22, 2016
150
38
28
JFYI, cross flash to P20 DOES work - use 9205-8e FW, as these cards do not support PCIe 3.0. However, be warned that the fan speed will increase to 70%. This happens with the latest HP P15 firmware as well, when using non-HP drives. I couldn't find a way around this, so I reverted back to the P11 firmware shipped with the system. Fan speeds back to normal. Oh well...
Hmm interesting. Thanks for the info. What is "normal" fan speed on your chassis?

I don't remember the fans running lower before I did the updates though. I was also running with no drives installed in the beginning and the sound was about the same. Going to give P11 a shot and see if it's less horrible.
 

Fredrik

New Member
Mar 28, 2016
5
1
3
Hmm interesting. Thanks for the info. What is "normal" fan speed on your chassis?

I don't remember the fans running lower before I did the updates though. I was also running with no drives installed in the beginning and the sound was about the same. Going to give P11 a shot and see if it's less horrible.
I'll need to check again when I get home, but IIRC, it was somewhere between 10-30%? It is now a jet-engine at idle, as opposed to a jet-engine at full blast. Try popping in a non-HP drive in one of the nodes to see if that really triggers it. I was doing my LSI firmware comparison with drives in through the whole process, so I cannot accurately comment on how it would behave with no drives.
 

raileon

Active Member
Jun 22, 2016
150
38
28
I took some pics of these mounted in the rack before I start filling it up with other equipment. The rack is a Dell 4220 Standard which is 1070mm in length. That little thing on top of the two s6500s is a Dell 1U to give you an idea of scale, especially in the back. I'm using the 3rd Party Rail Kit for these minus the front brackets.

In the front shots you can see two QSFP cables plugged into the FlexLOM slot of the right node. Those cables should be the biggest thing coming out of the front that adds any length to the unit.


There are about 3 inches of space between the door and the front of the nodes. The blue Mellanox pull tab just touches the grill.


The dell is tiny in comparison. I'm probably going to end up putting these at the very top or bottom of the case so I don't have to jump over them with cables for other boxes.


There are about 2 inches between the back door and the fan tabs. I could've pushed these further back into the rack if I had needed the space up front but I'm not seeing the need.


Getting these in there wasn't anything special really. They're the size of a boogie board but they fit a normal rack just fine.
 

cookiesowns

Active Member
Feb 12, 2016
234
83
28
28
dang it raileon, really made me butthurt for missing the deal. Would fit perfect in a 1200mm deep rack.
 
Aug 2, 2015
80
31
18
42
raileon,

You mean it could fit a 1000mm depth rack and still able to close the doors?! Sigh, I guess I have to rip apart my wallet now.
 

cellcreator

New Member
May 16, 2016
3
0
1
37
Hello,

I also ordered one unit of this machine and it arrived today. Unfortunately, the packaging materials were completely damaged, and some stuff might be missing.

Could anyone provide a list of items delivered with the chassis so I can contact with the seller about missing items? Besides, the top right node is a little bit difficult to remove and install. I'm worried if any of the nodes may fail running. Could anyone give me some advice on dealing with such situation?

Here are some photos of this terrible shipping:
Dropbox - HP_Proliant

Thanks!
 

raileon

Active Member
Jun 22, 2016
150
38
28
Hello,

I also ordered one unit of this machine and it arrived today. Unfortunately, the packaging materials were completely damaged, and some stuff might be missing.

Could anyone provide a list of items delivered with the chassis so I can contact with the seller about missing items? Besides, the top right node is a little bit difficult to remove and install. I'm worried if any of the nodes may fail running. Could anyone give me some advice on dealing with such situation?

Here are some photos of this terrible shipping:
Dropbox - HP_Proliant

Thanks!
That firs pic is how it showed up at your door? I think everyone got the same crappy packaging but the sticky pinkish shrink wrap stuff was on tight.

As far as parts, you seem to have everything. All nodes, the rail kit is the 2 front parts, the skirt for the back and the bag of screws. I'm assuming all your fans are there, you even got a custom wood handle on one of them. Did you get your heat sinks in a separate delivery?

A couple of my nodes are a little hard to put in and slide out, but I doubt it was from shipping since mine were locked on tight to a pallet. The chassis itself just isn't completely rigid.

You should definitely contact the seller with pics of how it arrived in case they want to file a claim with the freight company. It also gives them plenty of notice that some of your nodes might not work and why that's the case.
 
  • Like
Reactions: cellcreator

Layla

Game Engine Developer
Jun 21, 2016
215
177
43
40
@Layla The sas cables can be bent enough to get the nodes in although it doesn't seem like the best way to handle it.
Anybody figure out a better way yet?

Are the 2 SATA ports Intel 6G ports from the chipset? I'm tempted to just use those for the moment.

I'm really shocked that there don't seem to be any 90 degree SAS couplers, or any HP brand cables with a 90 degree SAS connector..
 

cellcreator

New Member
May 16, 2016
3
0
1
37
@raileon Thanks for the reply. Yes, the first picture shows exactly how it arrived at my door...

I just called the seller today. I was told the heatsinks were missing and will be sent right away, arriving next day. The seller asked me to test the server but he said it should be functionally fine. If there are some not working, some amont of refund can be issued. He also told me so far for this batch of machines, only one user reported that one fan was not working; and in another case, one of the nodes was difficult to remove and install. My top right node is a little bit difficult to remove but seems not a big deal. The seller was very nice and patient to answer my questions.

I'll need to test the server in the next a few weeks.

That firs pic is how it showed up at your door? I think everyone got the same crappy packaging but the sticky pinkish shrink wrap stuff was on tight.

As far as parts, you seem to have everything. All nodes, the rail kit is the 2 front parts, the skirt for the back and the bag of screws. I'm assuming all your fans are there, you even got a custom wood handle on one of them. Did you get your heat sinks in a separate delivery?

A couple of my nodes are a little hard to put in and slide out, but I doubt it was from shipping since mine were locked on tight to a pallet. The chassis itself just isn't completely rigid.

You should definitely contact the seller with pics of how it arrived in case they want to file a claim with the freight company. It also gives them plenty of notice that some of your nodes might not work and why that's the case.
 

Zankza

Active Member
Feb 1, 2014
79
15
28
Anybody figure out a better way yet?

Are the 2 SATA ports Intel 6G ports from the chipset? I'm tempted to just use those for the moment.

I'm really shocked that there don't seem to be any 90 degree SAS couplers, or any HP brand cables with a 90 degree SAS connector..
As time passed, The SAS isn't tricky anymore, Bending has stayed in shape and I no longer worry or have issues taking out/in the nodes anymore.

You'll just have to cringe as you bend them real hard for first time but afterwards it's all good.
 

Zankza

Active Member
Feb 1, 2014
79
15
28
To everyone who bought this chassis/server:

How has the servers been working out for you?
 

raileon

Active Member
Jun 22, 2016
150
38
28
The servers have been working well so far. I still need to put a lot of work into them to get them running as distributed storage and/or distributed DB. I'm not sure what the final form will be as far as the software, I plan to experiment a lot there.

I've had the hardware setup for a while now. All blades are running dual cpus and are fully loaded with dimms. They all have those great mlnx network adapters in the flexlom slot that a few of us picked up around the time of the original deal.

I did end up with one bad board where adding a second cpu caused post to fail every time. The seller was great though and refunded me $80 for it. I picked up 2 replacement motherboards for $100 from alan tech and it was no problem switching the bad one out. I just added the previous serial and the ilo advanced license and everything worked perfectly. Right now I'm thinking I might have a second bad board since one of the blades keeps dropping 2 dimms during boot. It seems random and switching the dimms hasn't fixed it so I'm not sure what's going on there.

I've been amassing an unhealthy amount of ssds for this project. Now I need to work out modifications to the cages where I plan to run 8 ssds so I'll be busting out the dremel there soon. I also have some blades where I want to run sas3 drives hooked up to LSI 9340-8i cards but I've yet to find a break out cable for those that works. I wasn't expected that to be so difficult to find...

Also on my todo list is a complete reworking of the fans to get the sound under control. I already have a game plan there but I keep pushing that project back in favor of other stuff I'm working on.

Overall though I'm very happy with the blades. They are really hard to beat from a price vs feature set point of view especially when you throw in those dirt cheap connect-x3 cards into the mix.

Forgot to mention: Two of my blades have sl250s stickers in the front. Thought that was odd since the sl250s chassis looks like it's one piece. Did someone really flip those stickers around? lol
 
Last edited: