Issue with Supermicro X8DTH-6F and L5639

[Mrlie]

Hi,

My OpenSolaris-server has been running on a Supermicro X8DTH-6F with a single E5620 for about 18 months, and I was contemplating buying a 2nd E5620 when I saw Patricks article about the L5639. Ordered two L5639 from eBay and received them last week.

On thursday I decided to do the upgrade, and I swapped out the single E5620 with the two L5639 and booted the server. I checked the BIOS that the two cpus were found and all looked nice and in order. But after POST it would not boot any OS.
I decided to boot OpenSolaris livecd and that went fine. Rebooted the server and now the OpenSolaris livecd didnt wanna boot properly. It starts the gui, but none of the icons or menues will show. I rebooted the server multiple times, and at one point the motherboard started to beep continuesly, which means system overheat, which I suspected was the CPU2 had perhaps gotten too much thermal compound.

I removed the 2nd L5639, but the same thing happend now; Didnt wanna boot from the SSD with the OS, or from the livecd.
I then decided to swap back to the old E5620, hoping for a fully functioning server like before, but to my surprice the server still acts the same. Redownloaded the livecd-iso and verified the MD5-checksum, but still had the same result.

Downloaded an Ubuntu livecd and that boots up fine on my E5620, have yet to try booting that on the L5639.
Ran Memtest+ for about 24 hours (almost 4 full runs) and no error were reported on the memory.

I then decided to restore default settings in the BIOS, and now I see the OpenSolaris boot-menu, but when attempting to boot the OS the server will either just freeze up or reboot within 20 seconds. Checked the BIOS to see why I suddenly could see the boot-menu, and it was because my SATA-setting was at IDE instead of AHCI. When I set my SATA to AHCI the boot-menu never shows up.

Have anyone experienced something similare? I've been building my computers for nearly 20 years, and running out of ideas.

 

[PigLover]

Your symptoms don't sound typical for a CPU fault - but the match almost exactly to memory error problems. Possible that you jostled the ram a bit when swapping CPUs? Try to run Memtest86+ and see what you get.

 

[Mrlie]

I moved 3 ram from the cpu1-slots to the cpu2-slots, but I also ran Memtest86+ for over 24 hours (completed almost 4 full runs for the 48GB I have), and that didnt rapport any error.
Then again many years ago I didnt get any errormessage from Memtest86+ until the 10th or 11th run.

I'll swap out the E5620 for the L5639's and also check the ram once again, that everything is ok and no dust/etc has managed to be wedged into the ram slots, and then let Memtest86+ run for a while.

 

[MiniKnight]

When you run a single proc are you running in the cpu 0 spot?

 

[Mrlie]

Yes, single cpu is in the correct slot. I'd say "offcourse", but I guess enough people have made that mistake over the years

 

[Mrlie]

Problem fixed!

Update:

Fixed the problem

Seems the AHCI-setting in the BIOS was at "Intel AHCI ROM" instead of "BIOS Native Module".
Super Micro Computer, Inc.. - FAQ Entry


Yesterday I replaced the E5620 with the two L5639 and then started Memtest86+ and let that run for little over 24 hours, almost completing 3 runs and detecting no errors.
At work today a thought occured to me, that I had been doing all the mounting of ISO-files through IPMI, but not physically on a dvd-rom. So I found a usb dvd-rom and that booted up the OpenIndiana livecd without any issues and all was working flawlessly. I then decided to reboot and change the SATA-setting from IDE to AHCI and see if the livecd would still show all my drives and SSD that I had the OS on.

It was then I actually noticed the two different AHCI-settings, and a quick use of google-fu I found the supermicro-faq linked above and everything became abit clearer to me
I changed the setting to "BIOS Native Module" and my OpenIndiana-installation booted right up without any issue.