Potential Deal: 2 x Dual 2011 nodes @$199, Quanta Openrack

[legopc]

I don't think they're actually for SAS/SATA, just that they use a miniSAS connector:

"The Efficient Performance motherboard has two PCIe* x4 external connectors on the motherboard. These connectors can be used to build a PCIe* connection between two systems.
The PCIe* x4 connector can be hot inserted and removed. A PCIe* 3.0 re-driver is used for PCIe* external links and supports a miniSAS cable up to 2 meters long."

I'll check the exact BIOS version and date tonight but from memory it was in 2012 for my Quanta, whereas I think someone earlier reported a version dated in 2013.

PCIe link between 2 systems??? how would that work and what would it be used for?

 

[e97]

I wonder how I could squeeze a decent GPU into the chassis to play it!

Is it possible to fit a GPU on the riser?

This could be a great data crunching platform.

 

[NorCalm]

If you look at the bottom photo:

Imgur: The most awesome images on the Internet

That's pretty much the size of the GPU available. Where the DIMMs start, there is a hard plastic cowling for air-exchange for cooling RAM/CPUs (which you can see on the second node in that photo).

What is the smallest GPU available? I can take a look at whether it fits.

Is it possible to fit a GPU on the riser?

This could be a great data crunching platform.

 

[e97]

If you look at the bottom photo:

Imgur: The most awesome images on the Internet

That's pretty much the size of the GPU available. Where the DIMMs start, there is a hard plastic cowling for air-exchange for cooling RAM/CPUs (which you can see on the second node in that photo).

What is the smallest GPU available? I can take a look at whether it fits.

Smallest would be a GTX950 / GTX960 but I was thinking at least a GTX970, if the only issue is the back hitting the shroud a PCI-E Flex Riser would allow the card to be just slightly above that:

 

[tuatara]

PCIe link between 2 systems??? how would that work and what would it be used for?

I imagine it would be used simply as a high bandwidth, low latency link? 3 separate PCIe 3.0 4x links at ~4 GB/s each, to the adjacent node or other nodes within the 2.0 meter cable spec.

Reading more of the Windmill specification it does actually state how SATA and SAS drives are supported through the miniSAS headers: 2 SATA ports to the motherboard SATA connectors, 4 SATA 3 Gbps to one miniSAS, 4 SAS 6 Gbps to the second miniSAS, and optionally another 4 SAS ports to the third miniSAS.

SATA
The motherboard has an Intel® C602 chipset platform controller hub on board. Two single SATA ports from Intel® C602 chipset SATA port 0 and port 1 are connected to two single SATA connectors, which support SATA 6Gbps. Four SATA ports from Intel® C602 chipset SATA port 2 to 5 are connected to one miniSAS connector, which support SATA 3Gbps. Four SAS ports from Intel® C602 chipset SAS ports 0 to 3 are connected to one miniSAS connector, which support SATA 6Gbps. When the upgrade ROM is installed, the Intel® C602 chipset will support four extra SAS ports (4 to 7) that are connected to one miniSAS connector.

The Efficient Performance board fully supports all 8 SAS ports. The HDDs attached to all the SATA connectors follow the spin-up delay described in section 9.3.3.

 

[NorCalm]

That's true - there's a slight concern on mechanical stability since using the riser would allow you to use the screws to secure to the chassis. But hey, it would be a lot of fun to see a GTX980Ti on this thing. I'm concerned about BIOS compatibility since the GT710 2GB had issues, though.

Smallest would be a GTX950 / GTX960 but I was thinking at least a GTX970, if the only issue is the back hitting the shroud a PCI-E Flex Riser would allow the card to be just slightly above that:

 

[NorCalm]

This is very interesting... is there any chance that there is this connection via the backplane between the two nodes? Or is that strictly power only?

I imagine it would be used simply as a high bandwidth, low latency link? 3 separate PCIe 3.0 4x links at ~4 GB/s each, to the adjacent node or other nodes within the 2.0 meter cable spec.

Reading more of the Windmill specification it does actually state how SATA and SAS drives are supported through the miniSAS headers: 2 SATA ports to the motherboard SATA connectors, 4 SATA 3 Gbps to one miniSAS, 4 SAS 6 Gbps to the second miniSAS, and optionally another 4 SAS ports to the third miniSAS.

SATA
The motherboard has an Intel® C602 chipset platform controller hub on board. Two single SATA ports from Intel® C602 chipset SATA port 0 and port 1 are connected to two single SATA connectors, which support SATA 6Gbps. Four SATA ports from Intel® C602 chipset SATA port 2 to 5 are connected to one miniSAS connector, which support SATA 3Gbps. Four SAS ports from Intel® C602 chipset SAS ports 0 to 3 are connected to one miniSAS connector, which support SATA 6Gbps. When the upgrade ROM is installed, the Intel® C602 chipset will support four extra SAS ports (4 to 7) that are connected to one miniSAS connector.

The Efficient Performance board fully supports all 8 SAS ports. The HDDs attached to all the SATA connectors follow the spin-up delay described in section 9.3.3.

 

[tuatara]

This is very interesting... is there any chance that there is this connection via the backplane between the two nodes? Or is that strictly power only?

No, the connection is over a miniSAS cable assembly. The motherboard to midplane connector has 4 power blades and 16 signal lines, but these are only SMbus, fan tachometer and some status signals.

 

[e97]

I'm concerned about BIOS compatibility since the GT710 2GB had issues, though.

I missed that! What issues did you notice with the GT710?

 

[smithse79]

Is anyone running one of these with the 10Gb SFP+ under linux? I installed Centos 6 and the only adapter I'm seeing is the 1Gb link. The mezzanine card is physically installed. There are references in the BIOS to Mellanox, but nothing outright stating that there is a NIC there.

 

[tuatara]

Is anyone running one of these with the 10Gb SFP+ under linux? I installed Centos 6 and the only adapter I'm seeing is the 1Gb link. The mezzanine card is physically installed. There are references in the BIOS to Mellanox, but nothing outright stating that there is a NIC there.

It is detected under FreeBSD 10.2 after I manually built the kernel module.

# uname -a
FreeBSD  10.2-RELEASE FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 15:26:37 UTC 2015     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

# kldload mlxen
mlx4_core0: <mlx4_core> mem 0xdf200000-0xdf2fffff,0xd0000000-0xd07fffff irq 42 at device 0.0 on pci6
mlx4_core: Mellanox ConnectX core driver v2.1 (Mar 11 2016)
mlx4_core: Initializing mlx4_core
mlx4_en: Mellanox ConnectX HCA Ethernet driver v2.1 (Mar 11 2016)
mlx4_en mlx4_core0: Activating port:1
mlxen0: Ethernet address: 00:02:de:ad:be:ef
mlx4_en: mlx4_core0: Port 1: Using 32 TX rings
mlxen0: link state changed to DOWN
mlx4_en: mlx4_core0: Port 1: Using 16 RX rings
mlx4_en: mlxen0: Using 32 TX rings
mlx4_en: mlxen0: Using 16 RX rings
mlx4_en: mlxen0: Initializing port

# ifconfig mlxen0
mlxen0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=d07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE>
    ether 00:02:de:ad:be:ef
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: no carrier

 

[tuatara]

I wonder how closely to spec all these Windmill OCP boards are... would the BIOS be interchangeable?! What is the version/date of the Quanta BIOS?

My Quanta has:

BIOS Vendor          American Megatrends
Core Version         4.6.4.1
Compliancy           UEFI 2.1; PI 0.9
Romley RC version    1.0.7
BIOS Version         F03_3A07
Build Date and Time  03/02/2012 13:08:57

The BIOS date is a few months before the build date on the assembly sticker so it hasn't been updated from the original.

What does your WiWyn have?

 

[svt3391]

On the Wiwynn:

BIOS Version         EP_P09RG.01 x64
BIOS Vendor          American Megatrends
Core Version         4.6.4.1
Compliancy           UEFI 2.1; PI 0.9
Romley RC version    1.0.0.18
Build Date and Time  04/09/2013 15:30:43
VR Version           2209

I have ordered a second unit which is in transit. I will check once it comes.

 

[YetAnotherMinion]

just make sure you plug the VGA1 monitor cable (not the VGA2). For Windows, the HD6450 is great and includes an HDMI/VGA/DVI trio, so you can get audio from the HDMI.

@NorCalm

What do you mean by VGA1? My google is failing me. Right now I have a Quanta Windmill, installed RAM and CPUs, and got a HD6450 graphics card. I see the blue light, then the yellow light comes on and goes off shortly, leaving the blue light shining steadily. I do not get any output on my monitor. I am using the VGA port (blue with 15 pins). What am I doing wrong?

 

[bcatx29]

Question to OCP Windmill's owners who installed Windows as primary OS: does your server successfully reboot/shut down when you use Windows default reboot/shut down method (Alf+F4, etc)???

I installed CentOS 7 on my wiwynn, I was having a hard time to restart the system, always need to do the reset manually.

 

[smithse79]

I am running Centos 6 and am having no issues using ACPI to shutdown/restart

 

[bcatx29]

I am running Centos 6 and am having no issues using ACPI to shutdown/restart

I can install CentOS6 on it, it halts and reboots nicely during the installation, but it won't boot up after that, stuck in entering grub menu, I've tried to install it from USB and PXE, same issue. How do you install your CentOS6?

 

[smithse79]

USB that I created a long time ago. I did see mention somewhere in this thread that you may not be able to boot from both USB ports. Try switching ports

 

[bcatx29]

Thanks for the tips, I'll try to install it again, might need to zero out the first couple MBs of the disk to clear the old mbr and table partition.

 

[tuatara]

Another data point, if it helps: I installed FreeBSD 10.2 to a SATA drive without a hitch using a USB stick on the internal port, a keyboard on the external port, and VGA using an ancient GeForce 7600GS that I had lying around.

Can you boot a live CentOS6 image?