Windows not seeing all SAS drives, anyone seen this?

[jcl333]

Hello all, thought I would start here because I am pretty sure this is some kind of Windows thing.

My Google-fu is not helping me much, too many articles about basic hard disk troubleshooting stuff.

Config:
SuperMicro X10SRH-CF (onboard LSI 3008)
16-bay SuperMicro chassis with LSI expander backplane connected to 1st port
Lastest BIOS, firmware, drivers, etc. (near as I can tell)
12x 4TB 6G SAS drives in chassis, 4x 400Gig eSSDs directly connected to 2nd port with breakout
Server 2019 DE on ESXi 7u1 with LSI passed through

Planning to play with SS with tiering (mirror accelerated parity)

I can always see all drives from the Broadcom storage management software, or from BIOS/UEFI but not from anywhere in Windows.
Not from disk manager, not from diskpart, not from powershell
Other OSs such as ESXi and TrueNAS also see all the drives with no problems.

Anecdotally it looks like Windows refuses to see more than about 11 of the drives no matter what I do. It will see the SSDs no problem.
I am planning to do some "process of elimination" type tests, because I have extra motherboard/SAS controller/cabling/drives/etc.
I will also try with Windows directly installed not virtual.

At first I thought I had a bad slot in the backplane, or bad drives, but some basic tests proved that not to be the case, and as I mentioned they are all visible from other methods.

Anyone run into anything like this?

Thanks in advance for any suggestions.

-JCL

 

[jcl333]

OK, so I solved the problem but still don't know why exactly it is happening.

- Booted off my local install SSD with Server 2019 DE on it, it has my collection of great toolz ;-)
- Pulled all the drives out
- Started with one drive that was not recognized - guess what? Not recognized regardless of the slot I put it in
- OK, start plugging all drives into one slot, some work some don't, and all the one's that don't won't work in any slot
- In the end I found about 7 drives that refused to show in Windows
- I have about 60 drives of these 4TB drives and I only need 12-16, so I went through more until I had at least 16 working one's
- Was able to successfully reach 16 working drives recognized in Windows
- I did have one scary moment, for a bit it looked like it would not recognize *any* drive in slot 0, but then I plugged an SSD into the slot and it worked fine. Then I tried simply switching the drives that were in slot 0 and slot 1, and then they both work.
- Another strange issue, if I refreshed the drive list in diskpart very quickly after inserting a drive (one that was OK) it would come up as "write protected". I could go into storage manglement, it would also show "read only" on that drive, but I could still delete the partition and then everything seemed normal. I suspect this is some kind of feature of SAS that protects the drive contents for a short time right after the drive is initialized when first inserted.

- Remember in all cases, these drives are always seen in the controller management software, UEFI BIOS< and the one's I tried also were recognized in FreeNAS/TrueNAS with no problems, as well as in ESXi v7u1 if I turned off pass-thru of the controller. Just Windoze refuses to see them at all.

- So, I guess all is well, if only some of the drives do this and it does not get worse, then whatever

- I suppose I could try to investigate what the differences are between the "bad" and "good" drives to try and figure it out. Maybe I will first try just re-running the low-level format that I initially did to change it from 520 to 512 byte sectors, maybe I should add some of those command switches I see that specifically remove protection information, I don't know.

- The one theory I have to explain this is, Windows keeps an internal database of disk signatures of sorts, it's how it uniquely identifies disks. I am sure others know how that works better than me. The place where I see that get used is when doing clustering and how it keeps track of shared disks. Doesn't quite explain why it "survives" another Windows installation that has never seen that disk before. So, this is just conjecture on my part. Every one of these drives was formatted using the Windows install I am now troubleshooting them with, so they did work and were recognized at one time. These are used drives that are a few years old, maybe some of them are just starting to die.

In any case, I would love to hear if anyone has a plausible theory as to what might cause this. Maybe there is a disk inspection tool that someone could suggest that could diagnose a possible problem. I did try a few that I have, but they could not see the drive either if Windows could not.

Still looks like a Windows problem, but of course something *IS* going on with these disks somehow.

-JCL

 

[i386]

I had drives that didn't show up in windows when they had some sort of meta data or (hardware)raid configuration on them.

Is your broadcom controller running it firmware?

 

[jcl333]

I had drives that didn't show up in windows when they had some sort of meta data or (hardware)raid configuration on them.

Is your broadcom controller running it firmware?

Yes, it is running IT firmware. It is possible there is something on them, I dunno.