Potential Deal: 2 x Dual 2011 nodes @$199, Quanta Openrack

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
K

kfriis

Member
Apr 8, 2015
54
7
8
51
@iguy

Success!

In the end, it was very simple. All i had to do was to manually enable PCIe bifurcation in the BIOS. Both you and I had settings on AUTO in the BIOS so I did not focus on this at first. However, force enabling bifurcation solved all my NVMe problems. I can now access my Intel 660p in Windows.

Hopefully this will help others with similar issues.
 
I

iguy

New Member
Feb 23, 2017
13
7
3
43
kfriis said:
@iguy

Success!

In the end, it was very simple. All i had to do was to manually enable PCIe bifurcation in the BIOS. Both you and I had settings on AUTO in the BIOS so I did not focus on this at first. However, force enabling bifurcation solved all my NVMe problems. I can now access my Intel 660p in Windows.

Hopefully this will help others with similar issues.
Click to expand...

Awesome!!! Bifurcation!? WT*! Lol! I'm glad it works! Quick question.

What exactly works ?
Can you see the PATA3 drive in the BIOS?
Can you see the EFI boot loader label in the bios? (shows "Windows Boot Manager" for windows)
Can you boot from it ?

Thanks
 
K

kfriis

Member
Apr 8, 2015
54
7
8
51
iguy said:
Awesome!!! Bifurcation!? WT*! Lol! I'm glad it works! Quick question.

What exactly works ?
Can you see the PATA3 drive in the BIOS?
Can you see the EFI boot loader label in the bios? (shows "Windows Boot Manager" for windows)
Can you boot from it ?

Thanks
Click to expand...
As expected, at lot of the things I tried along the way turned out not to matter. For example, installing the Intel NVMe driver was not necessary. In fact, I believe if I had just enabled bifurcation right away without any other changes, I am pretty sure it would have worked right away.

So far, I have tested Windows Server 2019 and VMWare ESXi v6.7 and everything works as expected. The NVMe drive shows up in Disk Management in Windows and can be manipulated just as any other storage device. Similar for ESXi.

As I mentioned, it was never my intention to boot from the NVMe drive. For that, I use a regular SATA harddrive for Windows and a USB drive for ESXi. I just need a fast storage device for a sizeable database (1.5TB+) that I am working with.

In the BIOS, under BBS Priorities (Boot order), the NVMe drive shows up as "PATA :SS".

The EFI boot loaders (including "Windows Boot Manager"), were there all along in the BIOS - that did not change when I enabled bifurcation. The only thing that changed was the appearance of "PATA :SS".

Question for you: is it worth updating from the B10 to the B11 BIOS? When I tried to update directly from my original B08 to your B11 BIOS, I received an error message, but maybe I can update to B11 from B10?
 
I

iguy

New Member
Feb 23, 2017
13
7
3
43
kfriis said:
As expected, at lot of the things I tried along the way turned out not to matter. For example, installing the Intel NVMe driver was not necessary. In fact, I believe if I had just enabled bifurcation right away without any other changes, I am pretty sure it would have worked right away.

So far, I have tested Windows Server 2019 and VMWare ESXi v6.7 and everything works as expected. The NVMe drive shows up in Disk Management in Windows and can be manipulated just as any other storage device. Similar for ESXi.

As I mentioned, it was never my intention to boot from the NVMe drive. For that, I use a regular SATA harddrive for Windows and a USB drive for ESXi. I just need a fast storage device for a sizeable database (1.5TB+) that I am working with.

In the BIOS, under BBS Priorities (Boot order), the NVMe drive shows up as "PATA :SS".

The EFI boot loaders (including "Windows Boot Manager"), were there all along in the BIOS - that did not change when I enabled bifurcation. The only thing that changed was the appearance of "PATA :SS".

Question for you: is it worth updating from the B10 to the B11 BIOS? When I tried to update directly from my original B08 to your B11 BIOS, I received an error message, but maybe I can update to B11 from B10?
Click to expand...
Cool, thanks for letting me know. Yeah the NVMe module definitely loaded properly. The "Windows Boot Manager"option is showing up from the SSD bootloader. Anyways i'm glad it works.

I strongly recommend installing the NVMe driver from Intel if you are running windows as host OS. The driver might have more features dealing with cache/firmware on the drive.(e.g Samsung 960 Pro - The cache is disabled if not using the Samsung driver, pain, i know..) Try running benchmark with/without the Intel driver.

On the BIOS update.. Honest answer, no. The change is minimal, if any. I noted some minor module upgrades and a couple of power/logging variables changed. Are you interested in performance ? Computing performance? Power saving?/Idle power saving? Disk I/O speed?
 
K

kfriis

Member
Apr 8, 2015
54
7
8
51
iguy said:
Cool, thanks for letting me know. Yeah the NVMe module definitely loaded properly. The "Windows Boot Manager"option is showing up from the SSD bootloader. Anyways i'm glad it works.

I strongly recommend installing the NVMe driver from Intel if you are running windows as host OS. The driver might have more features dealing with cache/firmware on the drive.(e.g Samsung 960 Pro - The cache is disabled if not using the Samsung driver, pain, i know..) Try running benchmark with/without the Intel driver.

On the BIOS update.. Honest answer, no. The change is minimal, if any. I noted some minor module upgrades and a couple of power/logging variables changed. Are you interested in performance ? Computing performance? Power saving?/Idle power saving? Disk I/O speed?
Click to expand...
Since you asked, I tested and was able to boot directly from the NVMe drive with Windows Server 2019.

I actually installed Windows as a VM/guest running in an ESXi host first (with the NVMe drive as a "passthrough" PCIe device). After verifying functionality in Windows as a guest, I then booted directly to Windows on the NVMe. That way, I can either boot ESXi and run a Windows guest with exclusive access to the NVMe OR I can boot directly to Windows without changing anything other than the boot device.

The NVMe now shows up as a EFI boot device ("Itntel xxx xxx") in tbe BIOS so I can just select it after pressing F11 during POST.

I also did some tests with the Intel NVMe driver installed or not. The tests are not conclusive. In most cases, the drive is a little bit faster using the Intel driver (as opposed to the built-in MS driver in Windows), but not in all cases. So depending on your workload, it is not a slam-dunk using the Intel driver.

My main goal is read performance on a database that is approx. 1.5TB. With 40 CPU cores, I will have sufficient CPU power, so I need to optimize I/O. My budget right now does not allow for more NVMe drives in order to run some kind of RAID, so I am trying to get as much read performance as possible for the money (some writes will also be done to temp tables etc. but not nearly as much as reads). For less than $200, I thought this NVMe drive would give me the most bang for the buck for my needs. Due to the size of the database, I don't think investing the same amount in RAM would be as efficient.
 
D

Dmytro Burianov

New Member
Mar 30, 2019
4
0
1
Hi All
I sow document with error codes from ipmi sol but I can not found it now
I bot 4 nodes, but start only 2 and first work good. i move CPUs(2*2609) and 1/3 ram to second node and it node do not start, I do not have video via external pcie card on monitor. On ipmi sol i see next codes in circle: [00][B6][30][15]
can you help me with it codes?
 
W

wladeeck

New Member
Apr 16, 2019
1
0
1
how to configure raid via sata? 2 ssd I try to connect, I select raid in bios, but the ctrl + i prompt to enter the raid array setup utility does not appear. I tried resetting the BIOS, different BIOS parameters, different ssd and hdd drives, different cables.
 
S

semidetached

Active Member
Sep 18, 2018
120
155
43
If anyone is interested in dual port 10Gb mezz cards for these, I have 3. They are interesting little cards (see ON 10GbE CX3 | OCP Network Mezzanine | QCT.io)
VGA on-board with basic KVM over IP

I also have 3 of the quanta boards with all of the MiniSAS connectors populated. Two are for SATA and one for SAS (if I remember correctly).

(Sorry if this is the wrong place to offer. I followed this thread when setting up my units and I ended up going a different way with all my LGA2011 equipment)
 
Last edited:
hmartin

hmartin

Well-Known Member
Sep 20, 2017
438
415
63
39
semidetached said:
I also have 3 of the quanta boards with all of the MiniSAS connectors populated. Two are for SATA and one for SAS (if I remember correctly).
Click to expand...
Nice, I didn't know any were manufactured with the connectors populated! I soldered the MiniSAS header for 4 additional SATA ports.

The other two MiniSAS are really SAS (see: page 6), but apparently an "Upgrade ROM" is required to use SAS ports 4-7.
 
R

RRB

New Member
Feb 5, 2019
7
10
3
Not sure how many Canadian's on here but i have a few Windmill's for sale on eBay with dual 2670's and 64gb ram. Someone could get a deal in the Toronto/Ontario area.
 
R

RRB

New Member
Feb 5, 2019
7
10
3
Does anyone still have a copy of the NVME boot modified BIOS? Link a few posts up is dead...
 
I

iguy

New Member
Feb 23, 2017
13
7
3
43
RRB said:
Does anyone still have a copy of the NVME boot modified BIOS? Link a few posts up is dead...
Click to expand...

I've uploaded it here:
BIOS.zip

Please refer to the thread for instructions. Make sure to read it carefully. And just in case, have a EPROM chip writer/reader just in case something goes wrong.

Check out the README file, i've updated it with new instructions. All 3 versions of the ROMs have the NVME driver.

Happy new year!
 
  • Like
Reactions: Nhan Nguyen
S

Scott Williamson

New Member
Dec 5, 2017
6
3
3
48
For anyone who is struggling to use a modern video card / GPU with the wiwynn sv7210 this mod to the VBIOS of the GPU worked for me link

The addresses were a bit different but it worked to change just two bytes as described. Now have VT-d pass-through of the GPU working and boot with a monitor attached.

Klee said:
I have had an issue with video cards mounted in the top slot, one a older Nvidia something 7200 Dell OEM the other is a older don't remember what ATI OEM Dell video card, I found that the Nvidia blue screens on windows 7 install and on Ubuntu 16.04.2 crashes when I try to install the recommended non-free nvidia 304 driver that shows up in synaptic but if I leave the free driver that installs originally it runs fine.

But if I take it out of the top slot and put it in the bottom slot of the riser it works perfectly.

So if any one is having a video card issue try putting it in the lower slot to see if that works, it seemed to work for me.

The ATI if I have it in the top slot the node never boots up at all and no video is displayed.

I'll try the ATI card in the bottom slot next.

I have noticed on the PCI-e riser there are two jumpers, does anyone know what they are for?

One is labled confgen1 and the other is confgen2.

EDIT: Its a Wiwynn server
Click to expand...
 
hmartin

hmartin

Well-Known Member
Sep 20, 2017
438
415
63
39
Mehmet said:
Did you try to connect hdd to the other sas headers? Do they work?
Click to expand...
There are three mini SAS connectors mid board. The remaining 2 mini SAS connectors as shown in the photo from semidetached are for PCIe x4 expansion (see page 6 of Open_Compute_Project_Intel_Motherboard_v2.0.pdf).

I discussed this previously: https://forums.servethehome.com/ind...-199-quanta-openrack.6856/page-32#post-167629

There are two headers for SAS connectors on the end of the board near the onboard 1GbE NIC, but I've never seen them populated. The third header (SAS 4-7) requires an additional option ROM to function.
 
M

Mehmet

New Member
Mar 26, 2016
13
1
3
52
hmartin said:
There are three mini SAS connectors mid board. The remaining 2 mini SAS connectors as shown in the photo from semidetached are for PCIe x4 expansion (see page 6 of Open_Compute_Project_Intel_Motherboard_v2.0.pdf).

I discussed this previously: https://forums.servethehome.com/ind...-199-quanta-openrack.6856/page-32#post-167629

There are two headers for SAS connectors on the end of the board near the onboard 1GbE NIC, but I've never seen them populated. The third header (SAS 4-7) requires an additional option ROM to function.
Click to expand...
I think I did not ask question clearly.

I know that minisas headers in front of motherboard next to ethernet ports are for NTB on PCIe.
I read all your posts about 3 storage ports. I would like to thank you that It was very helpful to me.
As you explained, there are three mini SAS connectors on the mid board. One for SATA ports 2-5 and others for SAS ports 0-3 and 4-7. We know that SATA ports 2-5 are working. Not sure about their bandwidth. I think it is 3Gbps.
According to manual, SAS ports 4-7 need upgrading ROM. But nothing about SAS ports 0-3. We can use them as additional 4 drives, total 10 but you reported that only SATA ports were working. My question was about SAS ports 0-3.
Drives are connected to SATA ports 2-5 at the picture loaded by Semidetach. My question was whether he tried to connect drives to SAS port 0-3, middle socket on the picture.
 
You must log in or register to reply here.
Top Bottom