Potential Deal: 2 x Dual 2011 nodes @$199, Quanta Openrack

[Tom B]

I am planning to set up four pairs in a cluster. Each node running ubuntu or any supported flavor of linux. Except for ability to ssh and program to communicate over socket, i do not need other network services.

For a developer who do not have experience in network setups may be someone can share configuration.

If i were to get the Quanta version with SPF+ interface. What do i need to get all nodes interconnected. Any pointers to the cheapest switch, cables would be nice. If it comes expensive i am thinking of getting the unused Wywnn version.

Do i need to worry about the BIOS issue of Wywnn for adding Intel 750 NVMe (as data disk), or GTX1080 (if possible).

Thank you

 

[voxadam]

Again, I'm not sure about signature verifications, but you can disable ME via the BIOS (check page 95 of the tech manual) and via a jumper (for forced BIOS upgrades).

My GTX950 would start booting enough to show the splash screen but then just hang there. Switching to an older AMD worked fine. That said, I was using the included risers which split the x16 into two x8 so that may have caused issues too.

Great!

A few more quick questions about the GPU situation. First, do you remember if you were booting in BIOS or UEFI mode? Second, were you trying to boot using the GTX as your primary, secondary, or only display adapter? And last, when you say it would show the splash screen do you mean the system firmware/BIOS splash or a splash screen from the GPU firmware?

Thanks so much for the great information; I really appreciate it.

--adam

 

[voxadam]

the tech manual

Is the Rackgo X F03A exactly the same board as the Quanta Windmill or simply very close? Do the system and BMC firmware images for the F03A work on the Windmill?

 

[voxadam]

Earlier I asked if the Windmill boards supported E5-2600v2 (Ivy Bridge) CPUs, and while I'm still not 100% sure about the Quanta machines I did notice that datasheet for the Wywinn SV7210 does state that both E5-2600 and E5-2600 v2 are supported.

I just thought I'd share incase anyone else is interested in a potential upgrade path.

 

[doh]

After viewing the datasheet I purchased the Wiwynn to use with 2660v2 chips. Unfortunately, they do not work. I contacted the seller and he said that it's due to an outdated BIOS, but as others have mentioned in this thread, they're impossible to find anywhere. I think the Quanta boards would actually support the v2s since it looks like the BIOS updates are obtainable. I might order the Quanta board to try it out, but has anyone here who updated to the latest BIOS tried them with the v2 chips?

 

[McSquizzy]

Hi all,

Just finished reading these 21 pages of super useful info, thank you all

DonJon's solution of using HP PSUs and just soldering the 12v is really awesome, thank you for pics and the writeup.

I'm wondering if you guys think it's possible to run 1 node alone without the midplane?

Cheapest barebones rack I could find was $220 w/ shipping. For a single mobo @ $90 w/ shipping, I could accomplish the same goal, as long as it's possible to hack the connect and spoof the fans somehow. The datasheet pinouts for where the midplane connects to the motherboard shows some fan inputs to the mobo, as well as some "SMBUS" business. Considering I don't really need both mobos or the PSU included in the "usual" purchase, I'm thinking I could hack the midplane connector and just get the mobo by itself.

Thoughts?

Also, completely irrelevant, but I'm eyeing some "SAMSUNG 4GB 2RX4 PC3-10600R M393B5170FHO-CH9" RAM. I'm thinking I could throw 1 stick in every 4 slots (4 sticks total per node), to start out with. I've never touched server RAM, so is it even possible to use only 4? Or do every other (ie. 8 total) slots need to be filled per node?

Thanks in advance; n00b questions from a college kid

 

[s2600gl]

@McSquizzy:

Open Compute WindMill v2 Motherboard

 

[grfxlab]

Hi all,

I've never touched server RAM, so is it even possible to use only 4? Or do every other (ie. 8 total) slots need to be filled per node?

Thanks in advance; n00b questions from a college kid

When setting up my machines, updating bios, and testing, i installed only two dimms. 1 for each processor.

 

[doh]

Happy to report that the v2 chips indeed work on the Quanta boards after the BIOS update!

 

[snake eyes]

Can anyone help with bios settings? I have supermicro servers that when setting power management to "performance"on e5-2670's it will run all cores at 3Ghz at 100% load and 68 degrees(although super loudly). I have been trying different settings on the Quanta and have gotten 2.7ghz on one cpu and 2.8 on the other. The cpus are running at 95 watts and temps for CPU 0 are 85 deg and CPU 1 are 76 deg. Is there something that can be set in the power management long or short duration or elsewhere to help squeeze more from these cpu's? The supermicro's process 30% faster with the same specs.

Hi gfxlab, did you fix the temperature problem. I have the same situation.

 

[Klee]

I was reading all 19 pages of the thread, and the links, at work today an saw the pics and info for the HP power supply mod, now I'm at home and can't seem to find it.... Could someone point me to the post please? Thanks!

"DonJon's solution of using HP PSUs and just soldering the 12v is really awesome, thank you for pics and the writeup."

 

[voxadam]

In the event that not everyone is comfortable with hacking power supplies, or simply would prefer a premade solution I found that there are adapters available for the HP Common Slot power supplies. It seems that the PSUs are quite popular with *Coin miners and someone decided to fill a void and market a fairly turnkey solution.

Original source:
GIGAMPZ Power supplies for Bitcoin & Ethereum mining, Industrial grade, Energy Efficient, unbeatable value.

Much cheaper eBay option:
Breakout Board Adapter for HP 1200 Watt DPS-1200FB Power Supply Bitcoin - US$26.98 eBay

EDIT: Oops, forgot to mention something.

I also found another adapter that was designed to power up to eight PCIe GPUs which I thougth may be of interest to someone.

Breakout Board Adapter for HP 1200w DPS-1200FB Power Supply GPU - US$29.00 eBay



DISCLAIMER: I have no association with either of these products, their sellers, or manufacturers nor have I used any of the products mentioned. I simply discovered them while doing some personal research and thought others might find the information useful.

 

[voxadam]

Happy to report that the v2 chips indeed work on the Quanta boards after the BIOS update!

That's great news. It's too bad that the Wywnn machines don't appear to have firmware available with support for the Ivy Bridge EP processors, even though their documentation statues otherwise, but it's good to hear that the Quanta boards will work with the more modern processors. I was reasonably confident of the support after reading the changelog included with the most recent firmware but I'm much more comfortable with third party confirmation.

Thanks.

 

[voxadam]

I was reading all 19 pages of the thread

Something just occurred to me. This thread was 21 pages long the other day but, as Klee said, it's now a mere 19 pages in length. Is it a common occurrence for threads to shrink on this forum? If posts were deleted it would explain why first Klee, and subsequently I was unable to locate the post(s) which brought the HP power supply possibility to light.

 

[Klee]

Well I do remember it was 21 pages earlier while I was at work, I was not logged in at work and am here at home so I thought it was because of that and did not think about it so I opened another browser and its only still 19 pages not 21 like before......

Something just occurred to me. This thread was 21 pages long the other day but, as Klee said, it's now a mere 19 pages in length. Is it a common occurrence for threads to shrink on this forum? If posts were deleted it would explain why first Klee, and subsequently I was unable to locate the post(s) which brought the HP power supply possibility to light.

 

[resham]

Something just occurred to me. This thread was 21 pages long the other day but, as Klee said, it's now a mere 19 pages in length. Is it a common occurrence for threads to shrink on this forum? If posts were deleted it would explain why first Klee, and subsequently I was unable to locate the post(s) which brought the HP power supply possibility to light.

Both the power supply mods posts seem to be deleted. If anyone still looking for the mods, the PSU Used on second post was 750W Cs He Power Supply Kit, you can search B0025B0FAW in amazon for its page.

The first and third data pins were soldered together with a 300K register in between, assuming to start the psu (Same as atx power supply). And large planes on left were used as positive and negative connections.

Connection on the power mid-plane / fan unit was soldered with 3 positive and 3 negatives serially. (From first post)

 

[iq100]

Both the power supply mods posts seem to be deleted.

Is this true? Who deleted it? Why? Without any attribution? Maybe another site might do that but NOT STH. I hope.

 

[resham]

Is this true? Who deleted it? Why? Without any attribution? Maybe another site might do that but NOT STH. I hope.

I have no idea. May be admins/moderators can look into it.

 

[resham]

New guide for using third party PSU

Background:

Unlike desktops and other servers, the motherboard and fan/psu unit does not switch on/off the PSU directly. Even when the nodes are off, the PSU is still on. Powering the IPMI / Remote management i guess.


The default PSU provides 12V DC. Data pins are used for LED on the disk trays and load balancing current using AC/DC input to PCU. If you are not using DC backup source, you can ignore the load balance. If you want the tray leds, you can use hobby female jumper cables to supply 5 volts. If your new PSU provides backup source active signal you can use that at A2 pin with similar jumper cables. For normal usage, you can do without any use of the data pins.

More info on default PSU : http://www.opencompute.org/assets/download/Open_Compute_Project_700W_450W_Power_Supply_v1.0.pdf



You can get any 750W+ 12V Output PSU from ebay or amazon. You might want to review it first to see if it is noisy. Once you have it, you can see its specs to find the +ve, -ve pins and POWER_ON pin.

If you are okay with simple soldering, HP Common PSU are ideal.





For regular ATX PSU



Connect #14 in 20 Pin and #16 in 24 pin to any Ground pin to start the psu.


For Server Part:




Simple hack would be using Bent Car fuse Pins with hot-melt glue to firmly attach the wires. If some one has any experience using pins with insulator outside that might fit the receptor, please suggest.

If PSU supports on/off like HP Common Slots, you can also use the rocker switch from ebay ($2 for a dozen), with the register along one end. For regular ATX, you don't need a register.

 

[resham]

OS Installation without graphics card

1. Install OS using different computer and plug in the hard disk to the node.

Tested with Ubuntu 16.04 Server. Use SSH to access and configure further. Before moving the hardisk, make sure the nic is setup. Was eno1 for the 1GbE and enp9s0 for the 10GbE.

If you see hard disk activity but can not connect, you can re-mount the hard disk on first computer and check the syslog for interface names or other errors.
​

2. Install using IPMI

OpenCompute - Ubuntu Wiki
Edit the grub command line on the installation USB to include GRUB_TERMINAL and GRUB_SERIAL_COMMAND.

Default username/password for SOL are USERID / PASSW0RD
Password has number 0 instead of O.

You might have to restart the nodes once connected over SOL to see BIOS and GRUB menu.

Above docs have full docs, here are few helpful commands:

• ipmitool -I lanplus -H 10.0.0.30 -U USERID -P PASSW0RD sol activate
• ipmitool -I lanplus -H 10.0.0.30 -U USERID -P PASSW0RD power cycle
• nmap -sP 10.0.0.0/24

Replace 10.0.0.30 with ip of node.
​