VMWare Mega Build

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Kryax

Member
Oct 14, 2017
44
1
8
Yea was reading up on Napp-it right now. I will try what you were recommending Rand since I have the stuff setup and booting as baremetal Freenas right now. Then I will mess around with passing the onboard controller although I would have to readjust my SATA-DOM's that are connected to that controller I believe. Those would definitely narrow it down to if the HBA is the cause of this or something else like the backplane.

I will probably have to mess with more of this over the next couple days since I have some other things I need to attend to but I appreciate your guys assistance!
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
Well the first step is identifying the cause and baremetal is the best way to do this.

If it works without HBA then its the HBA (or the backplane if you always combine those)
If you can test the HBA without the backplane or the backplane with the onboard ports (both require specific cables you might not have at hand) then you can rule out some more...

If its not cables (which you replaced) it sounds like the backplane - not sure if you have been able to rule it out ? have only skimmed the thread...
 

Kryax

Member
Oct 14, 2017
44
1
8
Awaiting for SATA power cables (current power supply does not have them). While I wait, I am tracking the following things to try:

1. Connect 2 HGST 8TB directly to motherboard SATA ports
- Current power supply does not have sata power connectors so I have a few 4 pin to SATA power on order.
- Bootup drives under baremetal Freenas install.
- Create 2 drive Stripe/Mirror and test for issues.

2. Test 2 HGST 8TB drives with ESXI and Freenas VM
- Current ESXI install is on SATA DOM
- Will install a fresh copy of ESXI on USB and disconnect SATA DOM
- Passthrough SATA host controller to Freenas VM
- Create 2 drive Stripe/Mirror and test for issues.
3. Test 2 HGST 8TB drives with 9300-8i card
- What type of cable would required to connect the 8TB drives directly? Would I want a reverse, forward, or sideband breakout? Supermicro eStore - Networking / SAS / SATA Cables
- Connect 2 HGST 8TB drives to 9300-8i breakout cable
- Test with both baremetal Freenas install and ESXI Freenas VM.

4. Test 2 HGST drives with BPN-SAS3-846EL1? (https://www.supermicro.com/manuals/other/BPN-SAS3-846EL.pdf)
- Not sure if I can even isolate the backplane. What cable would this even require if it could be done?
- Connect 2 HGST 8TB drives to BPN-SAS3-846EL1 breakout cable?
- Test with both baremetal Freenas install and ESXI Freenas VM.
5. Test 24 HGST 8TB, 9300-8i, and BPN-SAS3-846EL1 with alternative OS (Napp-it. Linux)
- Will research more into this in the meantime.
6. Test 2 HGST 8TB drives with 9211-8i card
- Connect 2 HGST 8TB drives to 9211-8i breakout cable
- Test with both baremetal Freenas install and ESXI Freenas VM.
7. Test 24 HGST 8TB drives with 9211-8i card and BPN-SAS3-846EL1.
- Connect all 24 HGST 8TB drives to 9211-8i to BPN-SAS3-846EL1 connected to 9211-8i card.
- Test with both baremetal Freenas install and ESXI Freenas VM.



Edit:

1. Method 1 works with no errors.
2. TBD. Could not toggle passthrough of the SATA controller.
3. Method 3 exhibited same errors previous described in earlier posts.
4. Not tested.
5. Not tested.
6. Method 6 works with no errors.
7. Method 7 works with no errors.
 
Last edited:

Kryax

Member
Oct 14, 2017
44
1
8
Update for a few items made above. Item 1 worked without any issues. Item 2 I could not toggle passthrough of the SATA controller. Could be driver related as I am just using naked ESXI drivers for this motherboard. I show it as "Intel Corporatation Lewisburg SATA AHCI Controller" but is greyed out.

Edit:

Also need to figure out what cabling is required to connect the 9300-8i to the 2 standalone drives and the backplane to the 2 standalone drives to isolate the issue. If someone can recommend the correct type of cables please let me know.

Supermicro eStore - Networking / SAS / SATA Cables
 
Last edited:

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
I assume you had found this chain of threads re passing through SATA: https://forums.servethehome.com/ind...int-ahci-controller-vt-d-successful-aio.8716/ ? Never done it so no idea whether grayed out issue is discussed there.

Supermicro MiniSAS HD to 4x SATA should cover "9300-8i to the 2 standalone drives" if they are SATA and not SAS
Supermicro Internal Reverse Breakout Cross-Over MiniSAS HD to 4 SATA might help to connect the backplane to the onboard SATA ports which I think you want to do here (to use backplane without HBA)? This understanding is based on @PigLover 's post here https://forums.servethehome.com/index.php?threads/home-server-build.18782/#post-182595
Me personally have never attached an Expander backplane to SATA ports (only -A) so not sure that works but seems to;)
 
  • Like
Reactions: Kryax

Kryax

Member
Oct 14, 2017
44
1
8
I assume you had found this chain of threads re passing through SATA: https://forums.servethehome.com/ind...int-ahci-controller-vt-d-successful-aio.8716/ ? Never done it so no idea whether grayed out issue is discussed there.

Supermicro MiniSAS HD to 4x SATA should cover "9300-8i to the 2 standalone drives" if they are SATA and not SAS
Supermicro Internal Reverse Breakout Cross-Over MiniSAS HD to 4 SATA might help to connect the backplane to the onboard SATA ports which I think you want to do here (to use backplane without HBA)? This understanding is based on @PigLover 's post here https://forums.servethehome.com/index.php?threads/home-server-build.18782/#post-182595
Me personally have never attached an Expander backplane to SATA ports (only -A) so not sure that works but seems to;)
Thanks so much Rand. Ordered the cables and also just got a 9211 HBA so I can really just nail down what the issue is instead of ordering one piece at a time. I will probably have to get to troubleshooting this again next weekend when everything arrives.
 

Kryax

Member
Oct 14, 2017
44
1
8
TLDR: 9300-8i card looked to be the cause of the issue. 9211-8i card works!

Ok latest update of the following methods from mentioned above but also some other options that I tried:

3. Test 2 HGST 8TB drives with 9300-8i card
- What type of cable would required to connect the 8TB drives directly? Would I want a reverse, forward, or sideband breakout? Supermicro eStore - Networking / SAS / SATA Cables
- Connect 2 HGST 8TB drives to 9300-8i breakout cable
- Test with both baremetal Freenas install and ESXI Freenas VM.

This option exhibited the exact same behavior as seen before. This also excluded the backplane even being connected. Tried multiple drives, swapping cables around, and even updating to various firmware versions but could not fix. Never got to testing ESXI as it failed under baremetal Freenas setup.

6. Test 2 HGST 8TB drives with 9211-8i card
- Connect 2 HGST 8TB drives to 9211-8i breakout cable
- Test with both baremetal Freenas install and ESXI Freenas VM.

This option worked!!! I did multiple transfers back and forth from the network share and did not get any errors. Also checked to see if UDMA errors increased under SMART for the drives and they stayed the same (UDMA was up and over 200 errors on one drive). I did not test under ESXI as I wanted to do a baremetal test with option 7 below first.

7. Test 24 HGST 8TB drives with 9211-8i card and BPN-SAS3-846EL1.
- Connect all 24 HGST 8TB drives to BPN-SAS3-846EL1 connected to 9211-8i card.
- Test with both baremetal Freenas install and ESXI Freenas VM.

Had initial issues with getting the new system to even boot just like I had with the 9300-8i card. Put 9211-8i into another system and erased rom and just installed P20 firmware only. Put 9211-8i card back into new server and booted no issue.So I decided to do a back to normal setup. Started with baremetal Freenas and was able to create a network share with 4 x 6 vdev (6 drives per vdev) and 1 pool. Baremetal had no issues with network transfers. ESXI bootup and passthrough of the device went fine and tested good also!

If I had to venture a guess, I believe that the firmware for the 9300-8i still has issues with latest Freenas/FreeBSD OS. Maybe in a few years it will work but for now I am probably going to move on. Hopefully this helps anyone else that may come across this though.
 
Last edited: