Can MPIO be achieved on JBOD with 9305-8e?

[Radian]

I'm struggling with a setup between HyperV host and JBOD chassis (Supermicro 837-E26), using a dual-port 9305-8e HBA Windows MPIO doesn't recognize a second path. I've been trying to get a response out of LSI/Avago/Broadcom whatever they call themselves now, but after a month still no joy. Lack of HBA tools doesn't help. All firmware flashed to latest, SAS support added to MPIO.

I know MPIO is working as redundant iSCSI drives appear with RR MPIO connections and two paths in mpclaim output.

I'm wondering if the issue is with the choice of HBA, while the chassis is dual expanders and controllers, the 9305-8e is a single SAS core card, is it possible to get dual paths from it?

 

[vanfawx]

Do you have multiple SAS expanders in your chassis? Do you have SAS disks? Or SATA disks with an interposer. Those are the only ones that support 2 paths.

 

[Radian]

Yep, SAS drives with the E26 dual backplane from SM. Physically I have everything capable for dual paths but I don't see two paths reported in MPIO.

From what I've read of other's with similar configs, they have two paths in MPIO, so my only conclusion is HBA choice.

 

[vanfawx]

Strange. Can you boot this into Linux or another OS to see if the drives come up as multiple paths? Your hardware choice as you said, should support this.

 

[Radian]

I've tried WS2016 TP5 but the same results. I will try Linux and see what it gives.

 

[cookiesowns]

The 9305 should be similar to the 9300-8i that I've used. I was able to do MPIO wide-port (8x), but required quite a bit of driver fiddling. Dual ported seagate 8TB SAS drives, on BE2C backplane ( SAS3)

I think your issue might be the SAS3->SAS2 conversion

Have you tried installing MSM and seeing what avago/lsi dashboard reports to your connection to the backplane?

 

[Radian]

The 9305 should be similar to the 9300-8i that I've used. I was able to do MPIO wide-port (8x), but required quite a bit of driver fiddling. Dual ported seagate 8TB SAS drives, on BE2C backplane ( SAS3)

I think your issue might be the SAS3->SAS2 conversion

Have you tried installing MSM and seeing what avago/lsi dashboard reports to your connection to the backplane?

Good call, I had assumed MSM was irrelevant as I'm using HBA not RAID controllers, but now I can see that the 9300 controller is reporting two sets of disks, so the controller is definitely seeing two paths.

Thanks for the idea, but I'm still stuck with why my windows cluster fails validation because it can't see two paths to the SAS drives?!?! From the HW perspective everything is working....

I've got Supermicro looking at this but they ain't proving that helpful, so please tell me more about this MPIO wide port setting?

 

[cookiesowns]

wideport and MPIO are different things. I was just explaining my config.

As for windows validation, do you have a screenshot? There was some specific settings on the windows and device manager side in order to get MPIO to work. Do you have the MPIO feature/role installed?

 

[Radian]

These are the settings for the drives, the first one is showing a iSCSI drive that is connected to the cluster and the second one is the same dialog but for a SAS drive. If it works for iSCSI, then I'd expect the SAS drives to work the same. I've enabled SAS drives in the MPIO config, which leaves me with a DSM issue as the HW seems to be working OK.

Are you aware of any SuperMicro DSMs required to get MPIO working?

 

[cookiesowns]

change the MPIO policy on the drive to round robin, not sure why it's not set to that by default. Try that, and reboot. DSM?

 

[Radian]

Ah yes, it was originally RR but I read somewhere on an Oracle site that such a config works only with LB. Anyhow, I'll change it back to RR.

DSM = Device Specific Module


Dell has one for their MD SAN, so I'm wondering if SuperMicro have one to enable dual paths.

 

[cookiesowns]

Ah yes, it was originally RR but I read somewhere on an Oracle site that such a config works only with LB. Anyhow, I'll change it back to RR.

DSM = Device Specific Module
View attachment 3099

Dell has one for their MD SAN, so I'm wondering if SuperMicro have one to enable dual paths.

Ah. That thing, i remember now. You need to install the LSI/Avago DSM's for your HBA.

 

[Radian]

lol, that was one of the things I looked for, but couldn't find anything remotely similar to DSM for LSI.

Any ideas where I might find it?

 

[cookiesowns]

lol, that was one of the things I looked for, but couldn't find anything remotely similar to DSM for LSI.

Any ideas where I might find it?

It's part of the Avago driver download package. It's in the .zip.

 

[Radian]

Thanks for all the help cookiesowns, I had a look at the driver zip but on loading the inf file into the DSM gives an error.

Now trying Broadcom support to see if they can give me the correct DSM.

 

[cookiesowns]

Thanks for all the help cookiesowns, I had a look at the driver zip but on loading the inf file into the DSM gives an error. View attachment 3108

Now trying Broadcom support to see if they can give me the correct DSM.

Not 100% sure if that's the right file. Check the readme if you haven't already, but your card is a newer chip, ( 16 port on single IC ) so could be having driver issues.

Let me know how far you get with broadcom support.

 

[elchimore]

Hi,
I'm maintaining a 2012 Sofs Cluster with quite similar setup..... If you want to access the JBod with two Clusternodes, you don't have a Multipath configuration. You have two JBods in one case each attached with one path to each clusternode. There is no need for a multipath setup. You only need MPIO setup if you connect the JBOD with two channels only to one host..... But then it's only local storage and no cluster storage, or are you trying to setup Storage Spaces Direct?
HtH Micha

 

[Radian]

Thanks elchimore, I started down this path as cluster validation gives me warnings about single path and yet my iscsi LUN in the same cluster gives me two paths, hence the confusion. My other issue that I'm experiencing (which I assumed was a MPIO issue), relates to failover of VM's and the inability to start once failed over. My VM's on iSCSI work fine during failover, but VM's on cluster SAS CSV fail during failover "saved state cannot be accessed" to be precise. I had assumed this was also caused by single paths in MPIO.

Have you seen anything like this on your config?

 

[elchimore]

Do you have the exact full Error message from the event log? Could be a problem with the access rights on the volume. There is a great german blogpost about setting up a SOFS Cluster and a HyperV Cluster:
Unsere Best Practice-Erfahrungen – Teil 1 – Die Installation eines Hyper-V Failover Cluster unter Windows Server 2012 R2 – Hyper-V Server Blog
Unsere Best Practise-Erfahrungen – Teil 2 – Die Installation und Einrichtung eines Scale-Out Fileserver unter Windows Server 2012 R2 – Hyper-V Server Blog
great posts, they are in german, but they are using english Windows versions Perhaps you can use it with google translator....
hth micha
(we've seen a lot of errors in our beginnings, the access rights are very very very important that all is working well..... it's the lesson I've lerned. And never copy files on the CSV by hand, use the funktion for moving the storage of the virtual mashine in the cluster manager, or some important rights will get screwed up)

 

[Radian]

Here is the error on failover - "Cluster resource 'Virtual Machine Dynamics 2012 R2' of type 'Virtual Machine' in clustered role 'Dynamics 2012 R2' failed. The error code was '0xc0370027' ('Cannot restore this virtual machine because the saved state data cannot be read. Delete the saved state data and then try to start the virtual machine.')."

Great site, works well through google translate