Potential Deal: 2 x Dual 2011 nodes @$199, Quanta Openrack

[bcatx29]

CentOS 6 is working fine now, I missed a "=" in my kickstart file, that makes it doesn't install CentOS bootloader correctly.

bootloader --location=mbr

No issue to restart or shutdown OS now.

 

[smithse79]

I'm curious as to why Centos 7 wasn't working

 

[bcatx29]

when I issue reboot command, it seems like hung after it shutdown all processes, I guess it's OS related bug or it's incompatible with motherboard's ACPI settings, no idea. I have 4 wiwynn systems, all have the same issue.

 

[smithse79]

Mine is a Quanta, I wonder if that makes a difference

 

[bcatx29]

They look identical, should be the same, just different brand.

 

[diglett]

Thanks to all the information in the thread, I picked one up from this seller on eBay. This one has the 2 slot PCIe riser instead of the mezzanine card and the power supply cables have an unusual three pin plug instead of a C14 inlet. I modified the cable with a 5-15 plug from Home Depot and used an extension cord to plug it into the ELC transformer from Amazon. I opted for the 1500 watt version of the transformer to be on the safe side.

I should have picked up a cheap video card to go with it, but it was possible to shoehorn in a 9600GT for long enough to install ESXi on a usb drive. Overall it's a tremendous value for the money and I'm really surprised at how quiet it is, though I haven't tried to stress the CPUs yet.

 

[Mehmet]

Hi,
I got 2 boxes of these and I think they are wywinn sv7210. I used hd3450 for video out and installed Centos 7. BIOS info is:
System Manufacturer Wistron
System Product Name Freedom
Board Manufacturer Wistron
Board Product Name Windmill-EP

For the last few days, I have been trying to set up and use IPMI but unfortunately I failed. I read all the posts at this thread and really confused because some says no IPMI, some says no BMC or others SOL for IPMI. I tried openipmi but it failed because of driver problem. So tried FreeIPMI but again could not manage to connect or get any information. DMIDECODE does not show any IPMI info. Ipmi-oem, ipmi-locate, ipmi-dcmi etc. all fail to connect locally or in-band. But I know BMC has a MAC address and read from system logs that BMC tries to get an IP from DHCP.Finally, I again check BIOS and saw following info(at the advanced tab):
DCMI Support: Enabled
Power restore policy: No change
Current Policy: Last State
Serial Over LAN: Enabled

Device ID Data
Self Test Result: Passed
Device ID: 50
Device Revision: 01
Firmware Revision: 02.17
IPMI Version: 02.00
Manufacturer ID: 000157
Product ID: 0b.02

I am about to give up IPMI/BMC issues. Is there anybody who succeed to power ON/OFF nodes remotely by IPMI/DCMI or just any idea?

 

[bcatx29]

Try to do this, Mehmet.

ipmitool -I lanplus -U USERID -P PASSW0RD -H xxx.xxx.xxx.xxx power on/off/reset
ipmitool -I lanplus -U USERID -P PASSW0RD -H xxx.xxx.xxx.xxx sol activate/deactivate

The "USERID" and "PASSW0RD" are the actual credential that I found on a manual for their other system. 0 = number zero

In the system's BIOS, go to SuperIO, and change the baud rate to 115200, UTF-8, that will give you a better output. Don't use the ipmitool on the Mac, it doesn't have lanplus option and will give you a lot of strange issues.

How's your CentOS 7, will you be able to halt/reboot it? I have 8 of them connected to ACUPWR 1500w, all of them cannot reboot/halt correctly, I have to reset/power off them manually.

 

[bcatx29]

Forget to mention when you are in the initial installation screen, press tab to insert serial console option to the kernel for redirecting the console output. In this way, you don't need a video card anymore.

console=ttyS1,115200

Hope this help.

 

[dba]

Here is a photo of four racks of these OCP servers that we just installed. Idle power per rack is around 28A 208V (4.6A on each leg of two 30A 208V 3-phase feeds) which really isn't bad for 104 Xeon E5 V1 CPUs, two switches, and a router.

BTW: The servers in the distant background belong to Sony and have something to do with the PlayStation. There must be two dozen racks at least- brand unknown.

 

[bcatx29]

I am so jealous... so one CPU each server and 64GB of memory?

 

[dba]

I am so jealous... so one CPU each server and 64GB of memory?

Each sled has two of the now super-cheap Xeon E5-2670 CPUs, 8x8GB RAM, and an eBay sourced Samsung 480GB Enterprise SSD drive. 480GB is too much, but they were dirt cheap and I'm planning on the extra capacity improving endurance since these servers hit the disk moderately hard. These servers run CPU-heavy but network-light tasks, so I removed the 10G Ethernet Ethernet mezzanine cards, saving some 550 watts per rack.

 

[Patrick]

@dba looking at that individual sitting next to the racks made me think Seinfeld a bit - is that a pick or a scratch?

Very cool pics. OH?

 

[dba]

@dba looking at that individual sitting next to the racks made me think Seinfeld a bit - is that a pick or a scratch?

Very cool pics. OH?

That's my Danish Ops Director there. Probably holding his nose to ward off the smell of the quirky OCP PXE behavior.

 

[Mehmet]

How's your CentOS 7, will you be able to halt/reboot it? I have 8 of them connected to ACUPWR 1500w, all of them cannot reboot/halt correctly, I have to reset/power off them manually.

Thank you for advices and I'll try them
I think my problem is about drivers and after little googling, found some solutions but i have not tried yet.Link is
Bug #1156667 “Need driver support for IPMI device on OCPv2 Windm...” : Bugs : The Open Compute Project

Centos 7 can shutdown system but reboot always hangs. I remember I read a bug report about reboot at the above site but could not find the link.

 

[ThisSpaceForRent]

That's my Danish Ops Director there. Probably holding his nose to ward off the smell of the quirky OCP PXE behavior.

Yeah..that was a pretty bad stench.....
Lesson 1 when working with these servers, buy a low end graphics card so you can see whats going on during boot and install.

 

[dba]

Yeah..that was a pretty bad stench.....
Lesson 1 when working with these servers, buy a low end graphics card so you can see whats going on during boot and install.

What model card worked for you? I know you were saying that some newer cards are too large and/or not compatible.

 

[ThisSpaceForRent]

ASUSCOMP HD5450 1GB SLNT LP PCIE
Had to remove the bracket but then it fit like glove.

 

[unwind-protect]

Two questions I didn't see answered so far:

1) does the BIOS support PXE boot? Can you enter "enough" of the BIOS to do it without the debug kit?

2) is the heatsink mount one of the socket 2011 standard formats?

Thanks!

 

[smithse79]

1) does the BIOS support PXE boot? Can you enter "enough" of the BIOS to do it without the debug kit?

Yes PXE is the default boot unless changed. If you find a GPU to put on it, it acts like any other system and you can get to the BIOS that way.