Sol 11.3: SAS3008 missing disk

[sundaydiver]

Hi

I have a puzzling problem with my Solaris 11.3 server.

I have added two new wd red disks to my SAS3008 hba IT mode, I can see both listed in sas3ircu:


but I only see one of them in Solaris. cfgadm -al shows as follows:


notice c4 shows as connected but unconfigured...
cfgadm -c configure c4 does not change anything.

What could be going on here???

 

[sundaydiver]

hmmm, maybe it is an issue with MPXIO being on by default?? Getting the following in /var/adm/messages with just the boot SSD connected to the HBA:

Sep 19 17:35:31 sonny-solaris genunix: [ID 408114 kern.info] /pci@0,0/pci15ad,7a0@15/pci15d9,808@0/iport@1 (lsc1) online
Sep 19 17:35:32 sonny-solaris scsi: [ID 583861 kern.info] sd2 at scsi_vhci0: unit-address g55cd2e404b700c23: f_sym
Sep 19 17:35:32 sonny-solaris genunix: [ID 936769 kern.info] sd2 is /scsi_vhci/disk@g55cd2e404b700c23
Sep 19 17:35:32 sonny-solaris genunix: [ID 408114 kern.info] /scsi_vhci/disk@g55cd2e404b700c23 (sd2) online
Sep 19 17:35:32 sonny-solaris genunix: [ID 483743 kern.info] /scsi_vhci/disk@g55cd2e404b700c23 (sd2) multipath status: degraded: path 1 lsc1/disk@w55cd2e404b700c23,0 is online

The SSD is connected to port1, and if I add another disk to port 2 it goes into maintenance mode Additionally, I can add another disk to port 3, but port 4 cannot host another disk . Seems like ports 1&2 and 3&4 are paired??

 

[sundaydiver]

yup, answering my own question.

disabled MPXIO for lsc driver devices and now all devices are available as expected!

 

[gea]

thanks for publishing

 

[whitey]

yup, answering my own question.

disabled MPXIO for lsc driver devices and now all devices are available as expected!

I have not ran into this but do intend to nab up some LSI 3008 ctrl's in the future. Can you post a mini how-to 'disable MPXIO for lsc driver devices' for the LMGTFY challenged today :-D

Just a config file edit or svcs/svcadm cmd?

 

[sundaydiver]

I have not ran into this but do intend to nab up some LSI 3008 ctrl's in the future. Can you post a mini how-to 'disable MPXIO for lsc driver devices' for the LMGTFY challenged today :-D

Just a config file edit or svcs/svcadm cmd?

I first tried the 'correct' way by commands to disable MPXIO on entire system:
- stmsboot -d

But that did not seem to work because afterwards my disks would still show up using:
- prtconf -vc /devices/scsi_vhci |grep dev_link.*s2

In the end it was just a config file edit to /etc/driver/drv/lsc.conf with line 'mpxio-disable = "yes"' (even though the file says "I/O multipathing (MPxIO) is enabled. Don't edit this property manually."

It seems to work...

 

[sundaydiver]

I first tried the 'correct' way by commands to disable MPXIO on entire system:
- stmsboot -d

But that did not seem to work because afterwards my disks would still show up using:
- prtconf -vc /devices/scsi_vhci |grep dev_link.*s2

In the end it was just a config file edit to /etc/driver/drv/lsc.conf with line 'mpxio-disable = "yes"' (even though the file says "I/O multipathing (MPxIO) is enabled. Don't edit this property manually."

It seems to work...

Ummm, yeah, don't do this! For some reason I am left with an unbootable server now after a simple reboot. Even though i had performed multiple reboots before now.
Stuck at reading zfs filesystems console prompt. No idea why, booted into single user mode and while mpxio still seemed to be disabled, it was back to having the disks located on multiple controllers. Tried stmsboot -u to see if it would update disks but resulted in stuck at boot with stmsboot reconfiguration prompt. only enough knowledge to break things more.

Think I will switch to 9211 hba, because these 9300 hbas don't seem to work too well at the moment. Although more likely I just don't know what I am doing.