Potential Deal: 2 x Dual 2011 nodes @$199, Quanta Openrack

tuatara · Mar 4, 2016

I can confirm that the miniSAS headers are missing on the Quanta boards too. The controllers are present.

The other differences that I've identified are that the WiWyn's are new, sealed in the box while my Quanta appears used (though very clean) and that the WiWyn includes a riser to split the 16x PCIe while the Quantas have a ConnectX-3 10 GbE in the mezzanine slot.

NorCalm · Mar 4, 2016

Thanks @tuatara!

I wonder how closely to spec all these Windmill OCP boards are... would the BIOS be interchangeable?! What is the version/date of the Quanta BIOS?

tuatara said:
I can confirm that the miniSAS headers are missing on the Quanta boards too. The controllers are present.

The other differences that I've identified are that the WiWyn's are new, sealed in the box while my Quanta appears used (though very clean) and that the WiWyn includes a riser to split the 16x PCIe while the Quantas have a ConnectX-3 10 GbE in the mezzanine slot.

tuatara · Mar 4, 2016

I don't think they're actually for SAS/SATA, just that they use a miniSAS connector:

"The Efficient Performance motherboard has two PCIe* x4 external connectors on the motherboard. These connectors can be used to build a PCIe* connection between two systems.
The PCIe* x4 connector can be hot inserted and removed. A PCIe* 3.0 re-driver is used for PCIe* external links and supports a miniSAS cable up to 2 meters long."

I'll check the exact BIOS version and date tonight but from memory it was in 2012 for my Quanta, whereas I think someone earlier reported a version dated in 2013.

legopc · Mar 4, 2016

tuatara said:
I don't think they're actually for SAS/SATA, just that they use a miniSAS connector:

"The Efficient Performance motherboard has two PCIe* x4 external connectors on the motherboard. These connectors can be used to build a PCIe* connection between two systems.
The PCIe* x4 connector can be hot inserted and removed. A PCIe* 3.0 re-driver is used for PCIe* external links and supports a miniSAS cable up to 2 meters long."

I'll check the exact BIOS version and date tonight but from memory it was in 2012 for my Quanta, whereas I think someone earlier reported a version dated in 2013.

PCIe link between 2 systems??? how would that work and what would it be used for?

e97 · Mar 4, 2016

NorCalm said:
I wonder how I could squeeze a decent GPU into the chassis to play it!

Is it possible to fit a GPU on the riser?

This could be a great data crunching platform.

NorCalm · Mar 4, 2016

If you look at the bottom photo:

Imgur: The most awesome images on the Internet

That's pretty much the size of the GPU available. Where the DIMMs start, there is a hard plastic cowling for air-exchange for cooling RAM/CPUs (which you can see on the second node in that photo).

What is the smallest GPU available? I can take a look at whether it fits.

e97 said:
Is it possible to fit a GPU on the riser?

This could be a great data crunching platform.

e97 · Mar 4, 2016

NorCalm said:
If you look at the bottom photo:

Imgur: The most awesome images on the Internet

That's pretty much the size of the GPU available. Where the DIMMs start, there is a hard plastic cowling for air-exchange for cooling RAM/CPUs (which you can see on the second node in that photo).

What is the smallest GPU available? I can take a look at whether it fits.

Smallest would be a GTX950 / GTX960 but I was thinking at least a GTX970, if the only issue is the back hitting the shroud a PCI-E Flex Riser would allow the card to be just slightly above that:

tuatara · Mar 4, 2016

legopc said:
PCIe link between 2 systems??? how would that work and what would it be used for?

I imagine it would be used simply as a high bandwidth, low latency link? 3 separate PCIe 3.0 4x links at ~4 GB/s each, to the adjacent node or other nodes within the 2.0 meter cable spec.

Reading more of the Windmill specification it does actually state how SATA and SAS drives are supported through the miniSAS headers: 2 SATA ports to the motherboard SATA connectors, 4 SATA 3 Gbps to one miniSAS, 4 SAS 6 Gbps to the second miniSAS, and optionally another 4 SAS ports to the third miniSAS.

SATA
The motherboard has an Intel® C602 chipset platform controller hub on board. Two single SATA ports from Intel® C602 chipset SATA port 0 and port 1 are connected to two single SATA connectors, which support SATA 6Gbps. Four SATA ports from Intel® C602 chipset SATA port 2 to 5 are connected to one miniSAS connector, which support SATA 3Gbps. Four SAS ports from Intel® C602 chipset SAS ports 0 to 3 are connected to one miniSAS connector, which support SATA 6Gbps. When the upgrade ROM is installed, the Intel® C602 chipset will support four extra SAS ports (4 to 7) that are connected to one miniSAS connector.

The Efficient Performance board fully supports all 8 SAS ports. The HDDs attached to all the SATA connectors follow the spin-up delay described in section 9.3.3.

NorCalm · Mar 4, 2016

That's true - there's a slight concern on mechanical stability since using the riser would allow you to use the screws to secure to the chassis. But hey, it would be a lot of fun to see a GTX980Ti on this thing. I'm concerned about BIOS compatibility since the GT710 2GB had issues, though.

e97 said:
Smallest would be a GTX950 / GTX960 but I was thinking at least a GTX970, if the only issue is the back hitting the shroud a PCI-E Flex Riser would allow the card to be just slightly above that:

NorCalm · Mar 4, 2016

This is very interesting... is there any chance that there is this connection via the backplane between the two nodes? Or is that strictly power only?

tuatara said:
I imagine it would be used simply as a high bandwidth, low latency link? 3 separate PCIe 3.0 4x links at ~4 GB/s each, to the adjacent node or other nodes within the 2.0 meter cable spec.

Reading more of the Windmill specification it does actually state how SATA and SAS drives are supported through the miniSAS headers: 2 SATA ports to the motherboard SATA connectors, 4 SATA 3 Gbps to one miniSAS, 4 SAS 6 Gbps to the second miniSAS, and optionally another 4 SAS ports to the third miniSAS.

SATA
The motherboard has an Intel® C602 chipset platform controller hub on board. Two single SATA ports from Intel® C602 chipset SATA port 0 and port 1 are connected to two single SATA connectors, which support SATA 6Gbps. Four SATA ports from Intel® C602 chipset SATA port 2 to 5 are connected to one miniSAS connector, which support SATA 3Gbps. Four SAS ports from Intel® C602 chipset SAS ports 0 to 3 are connected to one miniSAS connector, which support SATA 6Gbps. When the upgrade ROM is installed, the Intel® C602 chipset will support four extra SAS ports (4 to 7) that are connected to one miniSAS connector.

The Efficient Performance board fully supports all 8 SAS ports. The HDDs attached to all the SATA connectors follow the spin-up delay described in section 9.3.3.

tuatara · Mar 4, 2016

NorCalm said:
This is very interesting... is there any chance that there is this connection via the backplane between the two nodes? Or is that strictly power only?

No, the connection is over a miniSAS cable assembly. The motherboard to midplane connector has 4 power blades and 16 signal lines, but these are only SMbus, fan tachometer and some status signals.

e97 · Mar 8, 2016

NorCalm said:
I'm concerned about BIOS compatibility since the GT710 2GB had issues, though.

I missed that! What issues did you notice with the GT710?

smithse79 · Mar 9, 2016

Is anyone running one of these with the 10Gb SFP+ under linux? I installed Centos 6 and the only adapter I'm seeing is the 1Gb link. The mezzanine card is physically installed. There are references in the BIOS to Mellanox, but nothing outright stating that there is a NIC there.

tuatara · Mar 10, 2016

smithse79 said:
Is anyone running one of these with the 10Gb SFP+ under linux? I installed Centos 6 and the only adapter I'm seeing is the 1Gb link. The mezzanine card is physically installed. There are references in the BIOS to Mellanox, but nothing outright stating that there is a NIC there.

It is detected under FreeBSD 10.2 after I manually built the kernel module.

Code:

# uname -a
FreeBSD  10.2-RELEASE FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 15:26:37 UTC 2015     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

Code:

# kldload mlxen
mlx4_core0: <mlx4_core> mem 0xdf200000-0xdf2fffff,0xd0000000-0xd07fffff irq 42 at device 0.0 on pci6
mlx4_core: Mellanox ConnectX core driver v2.1 (Mar 11 2016)
mlx4_core: Initializing mlx4_core
mlx4_en: Mellanox ConnectX HCA Ethernet driver v2.1 (Mar 11 2016)
mlx4_en mlx4_core0: Activating port:1
mlxen0: Ethernet address: 00:02:de:ad:be:ef
mlx4_en: mlx4_core0: Port 1: Using 32 TX rings
mlxen0: link state changed to DOWN
mlx4_en: mlx4_core0: Port 1: Using 16 RX rings
mlx4_en: mlxen0: Using 32 TX rings
mlx4_en: mlxen0: Using 16 RX rings
mlx4_en: mlxen0: Initializing port

Code:

# ifconfig mlxen0
mlxen0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=d07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE>
    ether 00:02:de:ad:be:ef
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: no carrier

tuatara · Mar 10, 2016

NorCalm said:
I wonder how closely to spec all these Windmill OCP boards are... would the BIOS be interchangeable?! What is the version/date of the Quanta BIOS?

My Quanta has:

Code:

BIOS Vendor          American Megatrends
Core Version         4.6.4.1
Compliancy           UEFI 2.1; PI 0.9
Romley RC version    1.0.7
BIOS Version         F03_3A07
Build Date and Time  03/02/2012 13:08:57

The BIOS date is a few months before the build date on the assembly sticker so it hasn't been updated from the original.

What does your WiWyn have?

svt3391 · Mar 11, 2016

On the Wiwynn:

Code:

BIOS Version         EP_P09RG.01 x64
BIOS Vendor          American Megatrends
Core Version         4.6.4.1
Compliancy           UEFI 2.1; PI 0.9
Romley RC version    1.0.0.18
Build Date and Time  04/09/2013 15:30:43
VR Version           2209

I have ordered a second unit which is in transit. I will check once it comes.

liqserv · Mar 21, 2016

Question to OCP Windmill's owners who installed Windows as primary OS: does your server successfully reboot/shut down when you use Windows default reboot/shut down method (Alf+F4, etc)???

YetAnotherMinion · Mar 22, 2016

NorCalm said:
just make sure you plug the VGA1 monitor cable (not the VGA2). For Windows, the HD6450 is great and includes an HDMI/VGA/DVI trio, so you can get audio from the HDMI.

@NorCalm

What do you mean by VGA1? My google is failing me. Right now I have a Quanta Windmill, installed RAM and CPUs, and got a HD6450 graphics card. I see the blue light, then the yellow light comes on and goes off shortly, leaving the blue light shining steadily. I do not get any output on my monitor. I am using the VGA port (blue with 15 pins). What am I doing wrong?

bcatx29 · Mar 23, 2016

liqserv said:
Question to OCP Windmill's owners who installed Windows as primary OS: does your server successfully reboot/shut down when you use Windows default reboot/shut down method (Alf+F4, etc)???

I installed CentOS 7 on my wiwynn, I was having a hard time to restart the system, always need to do the reset manually.

smithse79 · Mar 23, 2016

I am running Centos 6 and am having no issues using ACPI to shutdown/restart

Potential Deal: 2 x Dual 2011 nodes @$199, Quanta Openrack

Member

New Member

Member

Active Member

Active Member

New Member

Active Member

Member

New Member

New Member

Member

Active Member

Active Member

Member

Member

Member

New Member

New Member

New Member

Active Member