HP SL6500 4U 8-Node (16 CPU) Cloud Servers | 256 Threads | 2,048 GB RAM ~$2000

William

Well-Known Member
May 7, 2015
785
250
63
63
I took some pics of these mounted in the rack before I start filling it up with other equipment. The rack is a Dell 4220 Standard which is 1070mm in length. That little thing on top of the two s6500s is a Dell 1U to give you an idea of scale, especially in the back. I'm using the 3rd Party Rail Kit for these minus the front brackets.

In the front shots you can see two QSFP cables plugged into the FlexLOM slot of the right node. Those cables should be the biggest thing coming out of the front that adds any length to the unit.


There are about 3 inches of space between the door and the front of the nodes. The blue Mellanox pull tab just touches the grill.


The dell is tiny in comparison. I'm probably going to end up putting these at the very top or bottom of the case so I don't have to jump over them with cables for other boxes.


There are about 2 inches between the back door and the fan tabs. I could've pushed these further back into the rack if I had needed the space up front but I'm not seeing the need.


Getting these in there wasn't anything special really. They're the size of a boogie board but they fit a normal rack just fine.
And here I thought I was crazy running machines like this in my family room, its nice to see other insane people do the same :)
 

raileon

Active Member
Jun 22, 2016
150
38
28
And here I thought I was crazy running machines like this in my family room, its nice to see other insane people do the same :)
No sir you are not. I actually just had a dedicated 220v line setup there so that I can run a bigger UPS to feed more equipment that has no business being in a home. :D
 
  • Like
Reactions: William

Zankza

Active Member
Feb 1, 2014
79
15
28
I was having this trouble whole day before I realized this topic is best place to ask as well:

One of the node is refusing to POST with status code of 0168 "Memory and QPI link initialization start" I have done a complete hardware swap between a working node, with further investigation that concludes the old 2012 bios is only source of difference between working node. The SSP doesn't want to upgrade BIOS as it did with other nodes, I don't know why, but I'm stuck with a node that needs to be upgraded via iLO, and for that I need difference file to able upload/flash.

The URL is located here: http://h20565.www2.hpe.com/portal/s...77937&sp4ts.sn=USE22575W8&sp4ts.pn=650047-B21

If anybody could grab the BIOS file, I would be forever grateful and add it into topic's drivers post.

Thanks!
 

Zankza

Active Member
Feb 1, 2014
79
15
28
except for the part that this seems to price out closer to $5k it is interesting.

where are these 250$ 128gb 2760 dual cpu nodes with a free fully kitted chassis?
I bought mines for 599$ as few others did:

https://forums.servethehome.com/index.php?threads/hp-s6500-and-sl230s-gen8-crazy-deal.10342

There was small chance that S6500 would be flooded into market, but it appears to be just one or two rack of S6500 was being dumped by yousendit.com.

With 599$ and the item (along the marked price) the numbers do add up.
 

lzvolensky

New Member
Jul 24, 2016
23
10
3
44
Zankza, BIOS file you need is available in cp027855.exe archive (Win2008R2 platform) available here Drivers & Software - HPE Support Center.

I got there from http://h20565.www2.hpe.com/hpsc/swd/public/readIndex?sp4ts.oid=5177939&swLangOid=8&swEnvOid=4064

You said you have SPP, this file was released 29 Feb 2016 so it is available in 2016.04 SPP and the newest one. I have unpacked this executable from SPP2016.04, it's 2.517.984 bytes long, and also unpacked the BIOS file itself cpqp7512.ee1 which is 4.201.836 bytes. This BIOS is available inside the executable archive.


Executable here : http://www.filedropper.com/cp027855 [you really don't need this one]
Bios here : http://www.filedropper.com/cpqp7512

I'm providing all the info so that you are able to verify it and/or process yourself. To be honest, I personally would not trust any guy from internet sending me .EXE files :) If there is any way how you can check MD5 checksums of these files, I highly recommend to do it - otherwise THE BEST option for you would be simply to open your SPP ISO image if it's April2016 or newer, and find cp027855.exe in \hp\swpackages directory. Run it, unpack to any directory of your choice and use ILO to update with BIOS file.
 

Zankza

Active Member
Feb 1, 2014
79
15
28
I have already tried that, I have tried uploading EE1 file into ILO and even see message such as "flashing" but the ROM never changes no matter what I do, including giving it hard resets, turning off server before performing flash, etc.

Which led me to think, there must be something wrong with file, or the procedure I am performing. Additionally I recently discovered this:

I was going over the BIOS revisions trying to find a some copy somewhere on internet and Look what I found:Drivers & Software - HPE Support Center.

"Addressed an issue where the system may experience a no boot condition on the reset due to a fatal error. This issue is typically seen as the server hanging at 20% progress in the Early Video Initialization."

This is exactly my problem, Too bad they didn't disclose what "fatal" error is.
 

lzvolensky

New Member
Jul 24, 2016
23
10
3
44
I believe this quotation has not too much to do with your inability to flash BIOS. At the same time I have to admit BIOS in the form of EE1 file seems weird, although I have no experience with SL6500.

What BIOS and ILO firmware you have now ? Version (ie. I31 BIOS) and date please. Maybe we'll find something between what you have and the latest versions.
 

Zankza

Active Member
Feb 1, 2014
79
15
28
Further analysis, I did a bios update on other working but yet used node, it went very smoothly and even had proper event logs like so:



Where on the broken node I see the "firmware upgrade started" part but never seen "firmware upgrade complete."

Additionally, I did the manual flash via iLO instead of SPP, one thing I noticed is that during the "flash" duration, is MUCH more longer than on the broken node, it's almost quickly finished with flashing but albeit no event log of "upgrade completed"

I think I may just have to deal with the fact I have a total-bricked board... =(
 

lzvolensky

New Member
Jul 24, 2016
23
10
3
44
aha :( sounds like broken EPROM in the background or something. If there is backup ROM, try to switch to it - as I say, I don't know this little platform. Checking Bios versions from 04/04/2012 you have :

Drivers & Software - HPE Support Center.

there's pretty hefty list of changes. While it doesn't make too much sense, I would carefully recommend to update ILO to latest possible version which was 2.44 when I checked ~2 weeks ago. Don't think it will make difference, but it won't hurt :

Drivers & Software - HPE Support Center.

in this ILO firmware, they only talk about "Auto Power On" in relation to SL platform. Who knows, have met things which were not in logs soooo many times - especially with HP updates.



I personally would :

1) update ILO to v2.44
2) switch to backup BIOS, if there is one
3) try to update with some intermediary steps IF YOU CAN, ie. BIOS Dec2013, then Mar2015, then latest Feb2016. I know it sounds weird.


Good luck buddy !
 
  • Like
Reactions: MiniKnight

frogtech

Well-Known Member
Jan 4, 2016
1,346
209
63
32
You guys who got this chassis/deal, how are you enjoying the system with only 4 SFF drives per node? I wanted to do highly converged stuff with this, I have a C6100 sitting around that I need to sell, which is the 24xSFF drives, and I can't figure out which one I prefer, the 6 drives and hot-swap of a front-loading chassis or the shear compute density of the S6500 for any and all lab needs.