Patrick

HP / HPE Moonshot Unofficial Reference Guide

barmalej

New Member
May 31, 2019
5
1
3
These OEM heatsinks are impossible to find. I talked to sellers on Alibiba, we can order a custom-made ones with heatpipes but the minimum order is 30
 

Scott Laird

Active Member
Aug 30, 2014
277
112
43
One thing I noticed on mine is that the M.2 retaining screws appear to be part of the heat sink retainer assembly. Without the heat sink, there's no way to actually keep M.2s in place. It should be easy enough to draw up a part and 3D print it.
 

adam_kf

New Member
Apr 17, 2017
17
5
3
36
One thing I noticed on mine is that the M.2 retaining screws appear to be part of the heat sink retainer assembly. Without the heat sink, there's no way to actually keep M.2s in place. It should be easy enough to draw up a part and 3D print it.
Yes, the rear bracket is actually a separate HPE SKU. Here's the SKU's as far as I've been able to tell:

  • 867458-001 or 864712-001 - M510 Heatsink Extended
  • 867511-001 - M510 Heatsink Backplate
Note: the backplate is available through HPE spare parts, so the only thing we need is the actual CPU heatsink assembly.
 

Scott Laird

Active Member
Aug 30, 2014
277
112
43
Wow, the pricing on that is kind of wild. A quick scan showed prices between $47 and $130 with no real consensus at all. I need 4, and even $47 is kind of steep for what it is. I mean, it couple probably build my own with an hour or so of measurement and cad work and 3 or 4 print-fit-revise iterations.
 

Scott Laird

Active Member
Aug 30, 2014
277
112
43
I fired up my EL4000 with one M510 installed, and eventually got it working. It wouldn't power on with both PSUs installed but only one plugged in; I'm not sure if that's a feature or a bad PSU. Once I fixed that, I was able to log in to iLO and poke around a bit. I was able to boot the Ubuntu installer using remote media, which was cool, but then it cut me off once it booted because I don't have an advanced ILO license. So presumably I'll have to install via the serial console, which looks straightforward enough. Just ssh to Administrator@<ILO IP>, log in with the same password, and go from there.

It runs around 40C with no heatsink while booting. I don't want to put any load on it until I have heatsinks, but it looks like it's installable for now.
 

Scott Laird

Active Member
Aug 30, 2014
277
112
43
Here's my first pass at a heat sink mount: scottlaird/HPE_M510

That's still *slightly* too thick; the next rev will probably be about 1mm thinner. I'm using one of these right now, and it fits into the slot fine, but it's obviously slightly thicker than it should be.

It's designed to use M3 heat-set inserts to hold the screws from the heat sink. It's specifically intended for a specific model of insert from McMaster-Carr, but anything similar will probably be fine. I've included M.2 mounting points for 2260, 2280, and 22110 where possible, using M2 screws. This rev doesn't include holes for the SATA M.2 port.
 

Attachments

Last edited:
  • Like
Reactions: adam_kf

Scott Laird

Active Member
Aug 30, 2014
277
112
43
I now have 4 M510s running in an EL4000 with a newer rev of my heatsink mount. I had to *really* shorten it in order to fit underneath the second DIMM slot, so it's not using brass thread inserts anymore, as every one that I could find was too thick. So the heat sink screws simply thread into the plastic. Not ideal, but probably the best that can be done for cheap. I'm working on getting a few sample heatsinks, but that'll probably take 2-3 weeks.

I updated the GitHub project that holds the STL for the mount. The README has a few more details, including a link to the Fusion 360 source for the mount.
 
  • Like
Reactions: adam_kf and Patriot

adam_kf

New Member
Apr 17, 2017
17
5
3
36
I'm working on getting a few sample heatsinks, but that'll probably take 2-3 weeks.
Awesome works Scott! I'm going to try to print on my end, see how I go. I picked up one of these and subsequently another 3 more recently. Looking forward to see how you go with the heatsinks as well.

What's the ordering process look like for them?
 

Scott Laird

Active Member
Aug 30, 2014
277
112
43
I found a few places online that do semi-custom heatsinks. Which is to say, they take stock heatsink stock and machine it down to match a spec. I have an automated quote from one of them (low $20s each, quantity 10), along with an estimate of 3-ish weeks. Hopefully I'll hear more next week. It looks like they're going to generate a part number for the heatsink that I'm ordering, so it should be simple for people to order their own. Or, if a bunch of people were interested, then a group buy would drive the price down, but even quantity 5 the price wasn't horrible.
 

adam_kf

New Member
Apr 17, 2017
17
5
3
36
Interesting. I found a few Alibaba sellers last night who look like they do custom jobs with heat pipes and all. Unsure of cost but willing to investigate there as well. I'd love to get my hand on a proper 867458-001 heatsink to reference, but they're impossible to find... anywhere...

Also, you mentioned you were working with your m510's inside an EL4000. This is all _without_ heatsinks? Are you using the 16 core variety? How are the thermals looking?
 

Scott Laird

Active Member
Aug 30, 2014
277
112
43
Oddly, the M510s (16 core) without heatsinks run fine, at around 40C when idle. But they boot fairly slowly, which makes me think that they're hitting thermal throttling limits in between spot checks. They work well enough so I can deal with OS installs and so forth while waiting for my heatsinks to arrive. Or maybe "exist" would be better?

I should probably throw this in here: installing Ubuntu without paying for an advanced iLO license is actually fairly easy. The problem is that the standard iLO stops doing console redirection once the OS has booted, which makes talking to the installer tricky. So here's what I did (once for each M510):
  1. Boot the node with an Ubuntu 20.04 server install USB drive using the iLO console redirection.
  2. In parallel to this, ssh into the iLO, using the same username and password used for the iLO web interface. So 'ssh Administrator@1.2.3.4'. Then type 'vsp' to start the virtual serial port connection. You can watch the POST while you wait.
  3. In the iLO Java console redirection window: Once the node finishes its POST interrupt the grub bootloader so it wouldn't automatically proceed to boot linux. Hitting one of the arrow keys will accomplish this.
  4. Hit 'e' to edit the settings for the Ubuntu install Linux boot option.
  5. Add 'console=ttyS0,115200' to the 'linux' command line. I actually replaced the 'quiet' option so I could verify that the serial port worked, but it probably doesn't matter.
  6. Hit 'control-x' to tell Grub to start booting Linux (there's a prompt for this on the screen).
  7. Now watch the serial/ssh window. You'll see the kernel boot, and then it'll be quiet for a minute while installer messages scroll on the iLO console window. After the installer finishes booting, it'll prompt for input on the serial/ssh window. You should be able to install like normal at this point.
  8. After installing, the default Ubuntu server config includes a serial console, but the VGA console is the default so you won't see all of the boot messages and may not be able to interact with it if it fails to boot correctly. You can edit the grub config to change this. Just follow any of the random web pages that discuss setting up serial consoles for Ubuntu (or really any Linux distribution; they're all going to be the same at this level).
 

Scott Laird

Active Member
Aug 30, 2014
277
112
43
HPE's website says that there is a "critical" iLO firmware upgrade to v2.61 from early 2021. Mine came with v2.54, from Jan 24, 2018. Unfortunately, the link that my iLO provided for HPE firmware updates wasn't helpful.

I was able to download the latest version from here:


Basically, I searched for "moonshot" and filtered to OS-independent software. This produced a 'ilo4_261_moonshot.bin' file that I could upload via the iLO webpage. Seems to work fine. Better than fine, actually: it adds HTML5 remote console support. No more Java!
 

Scott Laird

Active Member
Aug 30, 2014
277
112
43
I have 5 custom heatsinks heading to manufacturing now. Hopefully they'll work on the first pass and I won't need to order a second (or third) batch. Quantity 5, it comes to about $35 each, but for 50+ it's under $10. That seems surprisingly reasonable for a custom heatsink.
 

adam_kf

New Member
Apr 17, 2017
17
5
3
36
exciting! i'm waiting with bated breath!

Fingers crossed for the "hole in one" first batch. I know the feeling... had a few custom engineering builds in the past (cases mostly) that were never "quite right" but nothing a Dremel and a drill couldn't fix :)
 
  • Like
Reactions: Scott Laird