Potential Deal: 2 x Dual 2011 nodes @$199, Quanta Openrack

[hmartin]

Hello,

I found this technical manual and it mentions an FSC Image Update procedure (page 203).

https://www.serverware.com.my/media/product_downloads/X300/Rackgo_X700_Manual_0528.pdf

Could somebody share the flashtool and the files if any?

There is also a note about flash recovering the ME FW on page 210.

Tomorrow I'll try it, because today I could reflash the bios, but the ME flash failed (write protected ... aborted after 5 retry).

Cheers

You can extract the ME firmware from the update file for F03C3A07 on Quanta's website (linked to earlier in this thread).

Hi,

I have the BIOS ver F03C3A07 and I don't see any options to control the fans (profile).

The CPU fans are not ramping up when the CPU temp is higher.

I already tried to change the fans and Px Therm Margin threshold through dcmi but it does not change anything.

It triggers an event but the fans stay calm.

Any idea where I could change that?

Thank you

As I recall there is no option in BIOS to control the fan profile because it's handled by the BMC, not BIOS.

Can you give the CPU temperature in deg C? Difficult to know how hot a "higher temp" is.

My CPUs (E5-2630L) regularly hit the mid-70s during memtest and the fans stay quiet.

I personally wouldn't worry about it. The E5 v1/v2 CPUs are only 32/22nm so electron migration is not a huge concern, and they will thermal throttle themselves if they determine the cooling is inadequate.

 

[Nz0]

You can extract the ME firmware from the update file for F03C3A07 on Quanta's website (linked to earlier in this thread).

I already tried to update the ME using their tool but it aborted: ME Update Read Data from ff000000 Not Match Retry.

Can you give the CPU temperature in deg C? Difficult to know how hot a "higher temp" is.

I aborted the stress test at 78C. Throttling is not an option. I need heavy load 24/7. Intel specs give a limit of 73C.
I swapped the heatsinks with noctua ones and quiet fans. as I need to operate it in a living room.

It would be nice to have the OEM-commands for the BMC. The SMTP ports are there. May be there is also a command for FSC.

Edit:I found that, but it is for another model. http://www.staroceans.org/e-book/S2B IPMI Commands.pdf

I also tried using ipmi command to change the thermal settings from -2 to -60, it generates events but the fans do not react.

My last option is to power the fan externally.

Any idea how i could flash it successfully (I have no external hw)? What about the flags mentioned in the README if I use flashrom?

I tried flashrom -p internal. It just confirms that a part is read-only.

I did not find the jumper 49 on the mobo. Edit:found between the CMOS Battery and the pci, but it did not change the result.

Tks

 

[Nz0]

At least I found a solution to have the fans at 100%. I neutralized the PWM cable on one CPU Fan. The board detects it as failed and it delivers 100%. Lol.

I also found these raw ipmi queries, but I cannot see what they do.

ipmi-raw 0x00 0x30 0x30 0x00 0x00
ipmi-raw 0x00 0x30 0x31 0x00

 

[hmartin]

At least I found a solution to have the fans at 100%. I neutralized the PWM cable on one CPU Fan. The board detects it as failed and it delivers 100%. Lol.

I also found these raw ipmi queries, but I cannot see what they do.

ipmi-raw 0x00 0x30 0x30 0x00 0x00
ipmi-raw 0x00 0x30 0x31 0x00

When you disconnect the PWM wire it's equivalent to setting the duty cycle to zero. IIRC the stock fan uses a negated PWM signal, so that would be the equivalent of 100% duty cycle to the fan. The board isn't kicking them into high RPM, that's the fan itself.

I mentioned before in this thread (I think) but you can buy a PWM controller and use it to set the fans to whatever duty cycle you want. Helps if you want something more than idle but less than full throttle.

The PWM controller even has a uart connection, so you can use a USB to UART and control it from the OS if you wish.

 

[Nz0]

When you disconnect the PWM wire it's equivalent to setting the duty cycle to zero. IIRC the stock fan uses a negated PWM signal, so that would be the equivalent of 100% duty cycle to the fan. The board isn't kicking them into high RPM, that's the fan itself.

I mentioned before in this thread (I think) but you can buy a PWM controller and use it to set the fans to whatever duty cycle you want. Helps if you want something more than idle but less than full throttle.

The PWM controller even has a uart connection, so you can use a USB to UART and control it from the OS if you wish.

Thanks a lot for your help.

Edit:For the full story I only neutralized one pwm fan and the mobo detects it at 0 RPM and kicks the other fan to 100%.

 

[kidrobot]

Humor, I accidentally sent an empty reply, newbe...... Putting in the content now.

So thanks to grxlab and his pointer to the quanta site. I happen to have the Quanta version of the server (bought from Essio) and I was successful in updating the bios which then enables installing Windows Server 2016 TP4.

A couple of notes, links:

The bios version was a little hard to find, wasn't 100% sure of which version of the machine I had and the website had two different variants. F03A and F03C. From my bios I seemed to have F03B......

Looking at the readme.txt's that were in each download I noticed that the F03C's readme had the following text "System hardware revision supported: F03B PVT board."

Taking a chance, I took the dos install option and was able to do the firmware update(s). (There is actually two, one for the bios and one for the "ME Firmware".)

After that, there was a new bios experience and it supports installing modern OS's.

Link to the bios/drivers/etc.

Download Center

Hope this helps others, appreciate the thread and it is nice to see such an affordable solution for home lab expansion.

Can you point me to the site for the bios download ? The link you provide doesn't work

 

[Nz0]

Can you point me to the site for the bios download ? The link you provide doesn't work

Download Center | QCT- Quanta Cloud Technology

 

[hmartin]

For anyone who is using non-stock PWM fans and is wondering why they're always screaming. The stock fans for the Windmill/Winterfell take an inverted PWM signal. So the board outputs an inverted PWM signal to drive the fans.

So a duty cycle of 100% would have the stock fans stopped, and a duty cycle of 0% would have them running full speed. If you replace them with normal PWM fans, they're getting the inverse of the duty cycle and thus running loud when they should be quiet.

I bought a CMOS inverter and put it between the PWM output of the Windmill and my fan. Good news, the duty cycle is now correct for the fan. Bad news, the inverted inverted duty cycle is so low that the fan doesn't spin...

Mirroring Nz0's experience, I ran a stress test and got the CPUs into the mid-70s and still the duty cycle didn't change enough to make the 120mm fan start. Not really good to have zero airflow.

So it looks like the best solution for me is to go back to using the external PWM generators I've discussed previously. I can set the 120mm fan to a fixed duty cycle and ensure that the CPUs don't cook themselves.

Here's the final product with the case I had mentioned in #611 cut out of MDF and spray painted black. Note to self: MDF is crap, don't use MDF again.

Really a shame that the fan profile in the stock firmware is so brain dead.

 

[Nz0]

Really a shame that the fan profile in the stock firmware is so brain dead.

So true. The box looks very sweet indeed.

 

[Syndroma]

The Chinese put additional fans between CPUs in a metal shroud, powered by one of the SATA power connectors.

 

[hmartin]

The Chinese put additional fans between CPUs in a metal shroud, powered by one of the SATA power connectors.

Using the original heat sinks or custom heat sinks?

Without a fresh air intake I don't see how putting fans between CPU1 and CPU0 would help anything. The major issue is that the air into CPU0 is already hot from cooling CPU1.

I wonder why they didn't stagger the heat sinks and put the RAM all to one side. Maybe the trace length would have been an issue preventing that.

 

[Syndroma]

Using the original heat sinks or custom heat sinks?

Original ones. Just increases the airflow through them.

 

[Nz0]

Using the original heat sinks or custom heat sinks?

Without a fresh air intake I don't see how putting fans between CPU1 and CPU0 would help anything. The major issue is that the air into CPU0 is already hot from cooling CPU1.

I wonder why they didn't stagger the heat sinks and put the RAM all to one side. Maybe the trace length would have been an issue preventing that.

Mine running at 100%.

Spoiler: Output sensors

Adapter: ISA adapter
Physical id 0: +56.0°C (high = +80.0°C, crit = +90.0°C)
Core 0: +56.0°C (high = +80.0°C, crit = +90.0°C)
Core 1: +53.0°C (high = +80.0°C, crit = +90.0°C)
Core 2: +56.0°C (high = +80.0°C, crit = +90.0°C)
Core 3: +54.0°C (high = +80.0°C, crit = +90.0°C)
Core 4: +54.0°C (high = +80.0°C, crit = +90.0°C)
Core 5: +56.0°C (high = +80.0°C, crit = +90.0°C)
Core 6: +55.0°C (high = +80.0°C, crit = +90.0°C)
Core 7: +54.0°C (high = +80.0°C, crit = +90.0°C)

coretemp-isa-0001
Adapter: ISA adapter
Physical id 1: +54.0°C (high = +80.0°C, crit = +90.0°C)
Core 0: +50.0°C (high = +80.0°C, crit = +90.0°C)
Core 1: +53.0°C (high = +80.0°C, crit = +90.0°C)
Core 2: +51.0°C (high = +80.0°C, crit = +90.0°C)
Core 3: +50.0°C (high = +80.0°C, crit = +90.0°C)
Core 4: +48.0°C (high = +80.0°C, crit = +90.0°C)
Core 5: +49.0°C (high = +80.0°C, crit = +90.0°C)
Core 6: +53.0°C (high = +80.0°C, crit = +90.0°C)
Core 7: +52.0°C (high = +80.0°C, crit = +90.0°C)

Spoiler: Output dcmitool sdr

Outlet Cntr Temp | 40 degrees C | ok
Inlet Temp | 37 degrees C | ok
PCH Temp | 51 degrees C | ok
P0 Therm Margin | -34 degrees C | ok
P1 Therm Margin | -36 degrees C | ok
P0 DIMM Temp | 48 degrees C | ok
P1 DIMM Temp | 49 degrees C | ok
HSC0 Input Power | 196 Watts | ok
HSC0 Input Volt | 12.46 Volts | ok
CPU0 Tjmax | 90 degrees C | ok
CPU1 Tjmax | 90 degrees C | ok
SYS_Fan0 | 0 RPM | cr
SYS_Fan1 | 1650 RPM | ok

The second fan per hs makes 2-3 C difference.

 

[kidrobot]

Hi
What did you use to power the board?

For anyone who is using non-stock PWM fans and is wondering why they're always screaming. The stock fans for the Windmill/Winterfell take an inverted PWM signal. So the board outputs an inverted PWM signal to drive the fans.

So a duty cycle of 100% would have the stock fans stopped, and a duty cycle of 0% would have them running full speed. If you replace them with normal PWM fans, they're getting the inverse of the duty cycle and thus running loud when they should be quiet.

I bought a CMOS inverter and put it between the PWM output of the Windmill and my fan. Good news, the duty cycle is now correct for the fan. Bad news, the inverted inverted duty cycle is so low that the fan doesn't spin...

Mirroring Nz0's experience, I ran a stress test and got the CPUs into the mid-70s and still the duty cycle didn't change enough to make the 120mm fan start. Not really good to have zero airflow.

So it looks like the best solution for me is to go back to using the external PWM generators I've discussed previously. I can set the 120mm fan to a fixed duty cycle and ensure that the CPUs don't cook themselves.

Here's the final product with the case I had mentioned in #611 cut out of MDF and spray painted black. Note to self: MDF is crap, don't use MDF again.

Really a shame that the fan profile in the stock firmware is so brain dead.

 

[Indecided]

The Chinese put additional fans between CPUs in a metal shroud, powered by one of the SATA power connectors.

That is one of the most atrocious mods i've seen in a while. I bought several dozen nodes with them - 80% of them were jury-rigged by cutting off the PWM lines and a few haphazard twists of the 12V and ground wires, and a bit of electrical tape holding them together.

Some of them were so close to the heatsink that I think after a few dozen hours running at high temperature and a gentle suction movement from the exhaust fan would bring the bare wires into contact with the heatsink and thus creating a nice big short.

Threw the lot of them right out. Sure, some had used proper connectors, but the majority couldn't have been bothered.

 

[Syndroma]

At least they can be thrown out. I still don't understand why the Chinese cut C14 power cord.

 

[Indecided]

They cut it beacuse the original cord is not a C14, it's the same head as the other side that plugs into the PSU. Only usable if you pick up the specific PDU. I've had to cut all of mine off too.

Perhaps the WiWynns come with a C14 but the Quantas, so far..are all the 3 pin molex-ish plug.

 

[Syndroma]

Didn't know that. My non-Chinese Quantas come with a C14. I only had to cut the Chinese ones.

 

[hmartin]

Hi
What did you use to power the board?

I'm using a DC-DC step down (12V->5V) to provide the PSU_PG signal, so I used 5V from this to power the PWM generator.

Since it also has a microUSB port for power, you can connect it directly to the internal USB port if you want a simpler solution.

 

[kidrobot]

Hi
Has each rack with 2 circuits of 30 A 208V?
Are you using racksolutions's rack?

Here is a photo of four racks of these OCP servers that we just installed. Idle power per rack is around 28A 208V (4.6A on each leg of two 30A 208V 3-phase feeds) which really isn't bad for 104 Xeon E5 V1 CPUs, two switches, and a router.

BTW: The servers in the distant background belong to Sony and have something to do with the PlayStation. There must be two dozen racks at least- brand unknown.

View attachment 1994