EU ebay: Supermicro X10SLE-F (Micro-Cloud node) usually under 10 Euro

[DanAnd]

@RageBone:
What exactly is in the 32mb of your dump? Just the jffs2 part?

 

[RageBone]

@DanAnd everything that is on the Mio Spi EEPRom.
I assume that is the jffs2.

Lets call it a full flash dump.

Flashed it to my other board with inaccessible ipmi and that is now reachable too.

 

[DanAnd]

Call it Stockholm syndrome! I bought a 3rd board! This time I have a nice CPU socket-cover and the seller used a quite large shipping box with no packing peanuts inside.

I ready did a backup of the BMC firmware, added 2x 4GB PC3-14900E in the blue slots and tried a Pentium G3220 and a Xeon E3-1220 V3, which both are working fine in my ATX Supermicro boards, which are also using the C224 chipset.

I get a Bootscreen and the Postcode runs up to 4.

Based on https://www.supermicro.com/manuals/other/AMI_BIOS_POST_Codes_for_Grantley_Motherboards.pdf this should be: testing Southbridge or maybe done testing Southbridge?.
It doesn't get further than this.

oh dear!

 

[hmartin]

I get a Bootscreen and the Postcode runs up to 4.

Based on https://www.supermicro.com/manuals/other/AMI_BIOS_POST_Codes_for_Grantley_Motherboards.pdf this should be: testing Southbridge or maybe done testing Southbridge?.
It doesn't get further than this.

Flash the latest public BIOS from Supermicro's website via SPI. In the extracted archive there is a 16777216 byte file called x10sle9.531, flash this directly to the SOP8 near the BMC flash. Take a backup first, but considering your board doesn't POST I don't think it's that important.

That's how I ended up resolving the error from the botched BMC BIOS update.

 

[nerdalertdk]

So lots of new information here, have anyone gotten an fully running system yet ?

Just ordered my cpu, just waiting on shipping

 

[hmartin]

Yes, you can reset the IPMI user by using the instructions I posted in #82, you'll need an SPI flashing tool. Then as long as your BIOS works and doesn't have a password, it should boot. If your BIOS doesn't POST or has a password, you can use the SPI flasher again to flash the Supermicro BIOS directly from their website. Be sure to clear CMOS after SPI flashing the BIOS but before powering up, or you may encounter issues booting.

You'll need java installed to use the remote console of the BMC.

Newer BMC versions (>2.19, though I've only tried 2.19 which came on the board and 3.81) don't appear to power up, at least on my board. I think Supermicro improved the backplane detection and the BMC doesn't detect that the blade is inserted, so it doesn't power up. But I haven't dug into this as RageBone suggested because I just haven't had the time.

Edit:
Power consumption is *very* low. I'm measuring 13W idle (E3-1220v3, 2x4GB PC3-10600E, 16GB boot SSD with OpenWrt x86) on the DC side. This is with the BMC booted and 2 Ethernet connections (BMC+primary NIC). The BMC alone seems to draw ~6W when the x86 side is off.

I'm going to go out on a limb and say you're unlikely to find a Xeon system that idles lower than this this thing does. With 4 SATA3 ports this thing would make a sweet NAS.

 

[DanAnd]

Power consumption is *very* low. I'm measuring 13W idle (E3-1220v3, 2x4GB PC3-10600E, 16GB boot SSD with OpenWrt x86) on the DC side. This is with the BMC booted and 2 Ethernet connections (BMC+primary NIC). The BMC alone seems to draw ~6W when the x86 side is off.
I'm going to go out on a limb and say you're unlikely to find a Xeon system that idles lower than this thing does. With 4 SATA3 ports this thing would make a sweet NAS.

And there is the famous E3-1220LV3 , which is just a dual-core Xeon, but with AVX and Hyperthreading and whopping 13W TDP. Sure it will be very slow compared to other Xeons, but maybe the exact processor for a low-powered NAS.
That is exactly my plan: The very small form factor screams for a small 3d printed NAS-case.

 

[DanAnd]

Flash the latest public BIOS from Supermicro's website via SPI. In the extracted archive there is a 16777216 byte file called x10sle9.531, flash this directly to the SOP8 near the BMC flash. Take a backup first, but considering your board doesn't POST I don't think it's that important.

Yes! The new board runs! Hurray!

I can confirm that the works really really fine
All 5 (!) SATA Ports also are working as expected.

Ensure that you have a static air flow. The parts are getting warm quickly.






Thanks everybody for the awesome support!

I will create a few how-tos, so that we have a central documentation on these.

 

[nerdalertdk]

Cool

yeah glad I bought the 2u cooler

 

[DanAnd]

My second board which also didn't want to start, just required the BIOS update, as @hmartin suggested.

So finally two of my boards are working just fine. The only one which I blowed up (I am pretty sure it was my fault) is still pending.

I can also confirm @hmartin 's statement:
If the BMC Firmware is updated to 3.2, the board boot anymore. Using the "old" (2.19) Firmware they are working just fine.

NOTE: Ensure that you make backups of the BMC-Firmware and your BIOS if possible. This will ensure that you will able to go back, if something goes wrong.

So what you need:

Solution 1:
* Flash-Programmer with SOP8 Connector to flash the updated BIOS
* Have a Console-Breakout Cable to be able to use VGA + Keyboard to install Linux to clean the IPMI ADMIN Account "locally".

After that, you can control the board via IPMI and don't need any console cable anymore.

Solution 2:
* Flash-Programmer with SOP8 Connector to update BIOS and SOP16 Connector to Read, Patch and Write the BMC-Firmware which sets back the IPMI User/pass to ADMIN/ADMIN
After doing that the Board will be reachable via IPMI and you can do everything remotely

Solution 3 (Not tested, but may be works):
* Flash-Programmer with SOP8 Connector to flash the updated BIOS
* build a PXE Boot Server ( LMGTFY ) which automatically installs a linux of your choice with a predefined user, so that you can connect to it remotely. If you are on it, you can clear the IPMI account "locally" in that linux shell.

Cheers,

Daniel

 

[hmartin]

Hey DanAnd, what's the speed of the 5th SATA port? The Supermicro specification for the board only mentions 4 SATA3 ports:

The 5th port is a different colour (at least on my board, the connector is black not white). Is it also on the same controller? (I know I could check myself, but since you already have 5 devices connected...)

Also, Supermicro doesn't seem to offer download of the older BMC versions, but this site appears to have all the versions archived, if anyone wants to test specific versions: ftp://ftp.abacus.cz/support/FW/MB/SUPERMICRO/IPMI/X10_REDFISH/

 

[hmartin]

Solution 3 (Not tested, but may be works):
* Flash-Programmer with SOP8 Connector to flash the updated BIOS
* build a PXE Boot Server ( LMGTFY ) which automatically installs a linux of your choice with a predefined user, so that you can connect to it remotely. If you are on it, you can clear the IPMI account "locally" in that linux shell.

It would probably be easier to take a hard drive with an existing Linux installation and with ipmitool installed, and just create a boot service to reset the IPMI user password

Or just a regular Linux installation configured with DHCP and SSH, then you can login and do the ipmitool commands interactively. PXE booting is hardcore

 

[DanAnd]

Hi,

Officially the black SATA port just for DOM devices. So something like a 4gb flash on a SATA plug. Nothing speaks against using it as a standard disk connector.
Regarding the speed, I can do a few tests on it and compare it with the 4 white SATA ports.

I powered all drives over my ATX powersupply and not via the onboard step down supply.

I will give the BMC versions a try to figure out the latest working version.

Cheers

Daniel

 

[DanAnd]

It would probably be easier to take a hard drive with an existing Linux installation and with ipmitool installed, and just create a boot service to reset the IPMI user password

My boards didn't boot from disks, but from PXE. So adding any drive with a bootable Linux doesn't help if you need to get into the bios first. So it is basically chicken and egg dilemma.

Easier it's of course to use the console breakout cable.

 

[azanotti]

Hi, could someone post of a picture of the entire power setup? I would like to know how difficult this board is to power before buying one. Thanks

 

[hmartin]

Hi, could someone post of a picture of the entire power setup? I would like to know how difficult this board is to power before buying one. Thanks

You need a 12V power source capable of providing at least 40W (for something like an E3-1220L v3) or the TDP of the CPU you plan to use + overhead (e.g. for an 84W Xeon v3, I'd recommend at least 120W power supply to be safe).

The power connector on the motherboard is the same physical connector as the standard ATX 12V EPS power. Pin 5 must be replaced with 5V stand-by. Tip: buy an EPS extension cable and modify the extension cable, with that you can use a normal ATX power supply with EPS output to power the board.

I am using an always-on 12V power source (such as the Dell DA-2) with a DC-DC step down converter to provide 5VSB. The total cost for a power supply + step down converter should be <20€ if you need to buy one.

RageBone has posted the required pin configuration on the jumper headers JP17 and JP18 to simulate backplane connectivity. Note that this method only appears to work for BMC firmware versions below releases 3.x

Powering the board is quite simple. Just provide 12V & 5VSB via the EPS connector to the motherboard, and you need 2 wires of 2.54mm type (Arduino/Raspberry PI GPIO style) to wire the required pins of JP17 & JP18. See GitHub for description/details of that.

Photo:

Note that several members have received boards with unknown IPMI logins and BIOSes that won't POST. For this reason I would highly recommend you invest in an SOIC8 and SOIC16 chip clip and an SPI programmer to reset IPMI and flash a newer BIOS. These will cost less than the Supermicro SUVI cable and are far more useful to recover your board should anything go wrong.

 

[azanotti]

Wow, thank you very much for the very extensive response. Will definitely try that out.

 

[nerdalertdk]

So just got my cpu and plug it all together, but I have a lot of coil whine!!

nothing seams to smoke or get hot

have any of you experienced this.

server.mp4

its hard to hear, but its there

 

[hmartin]

So just got my cpu and plug it all together, but I have a lot of coil whine!!

nothing seams to smoke or get hot

have any of you experienced this.

1. Do you have IPMI access? What does IPMI have to say about the state of the server?
2. Do you have a way to measure power consumption?

I don't have any coil whine when the board is off, IPMI alone consumes around 6W but nothing will get warm to the touch.

There is some coil whine on my boards when the x86 side of the board is powered, and you can expect the PCH heat sink to become warm to the touch after a few minutes. Power consumption should be between 20-35W depending on whether your board is passing POST or not.

Debugging is much easier if you have IPMI access, troubleshooting blind is very difficult.

 

[nerdalertdk]

Did not hook it yup yet, waiting on my SOP8 and SOP16

have not measured power yet, the coil whine on first boot just worried me


So..

You can't upgrade the BIOS to latest version with latest version of IPMI, but when I downgrade IPMI to 226 i could upgrade the BIOS to x10sle9.531, but its now in an post boot loop