LSI 9300-8i SR-IOV
- Thread starter ivans89
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Who says that SR-IOV is in any way supported? What would it even mean for an HBA? Google shows me a lot of talk about SR-IOV with LSI HBAs, but very little in terms of anyone acquiring the mythical firmware that enables it, much less what the VFs would expose and how.
In theory it would be a nice thing to be able to "pass through" the controller and some drives to something like TrueNAS while the rest of the drives are used for the hypervisor to run other VMs.
But, I can't imagine how the UI inside of a hypervisor would allow it to work. Would you pick a few drives, which would then lock them to that VM, and automatically virtualize the controller, too...essentially a raw device mapping on steroids? I don't want to think about the carnage that would happen if you just virtualized the controller and allowed multiple VMs and the hypervisor to access all the disks directly.
But, I can't imagine how the UI inside of a hypervisor would allow it to work. Would you pick a few drives, which would then lock them to that VM, and automatically virtualize the controller, too...essentially a raw device mapping on steroids? I don't want to think about the carnage that would happen if you just virtualized the controller and allowed multiple VMs and the hypervisor to access all the disks directly.
I think there are just too many moving parts for this to really be viable outside of a niche use case. Sure, you could assign disks to VFs, but then what about hot swap? What if you add a whole expander full of disks? Do you rely on SES to maintain a physical slot mapping table that can then be used to look up which VF to use for a given slot? What if you replace an expander full of disks with a different expander with as many disks? Now implement all this in a UEFI OpROM plugin menu and watch how users lose 20% of their hairs by the time they've figured out what the hell is even going on.
so the theory of how this works is you would get an SR-IOV device for each port on the HBA. so if you had an 8 port card you would have 8 LSI-9200-SR-IOV devices.
and it looks like there MIGHT have been a publicly available firmware for some subsection of 9200 devices at one time. but any internet chatter after the 'hey this could be a thing' posts pretty much disappeared. Though there is probably a group of Russians with a hacked modded fully fleshed out deployment running in some Russian datacenter.
and it looks like there MIGHT have been a publicly available firmware for some subsection of 9200 devices at one time. but any internet chatter after the 'hey this could be a thing' posts pretty much disappeared. Though there is probably a group of Russians with a hacked modded fully fleshed out deployment running in some Russian datacenter.
Ok. What happens when you add an expander? An expander is always going to be taking at least four ports. Is the card just going to hot plug a truckload of new PCIe devices? Does it remove three channels and shove every disk behind the expander into the first channel?
The concept breaks down very quickly, unfortunately.
There's a few ways it could be done. It could just require the host to pick which VF unassigned drives should be passed into, at the time when the drive is connected. Then, the hypervisor would be able to apply its own logic. This would make it effectively the same as software-based SCSI passthrough, but with better performance.
It could also just pass all the drives into every VF (and the PF). This would only be viable for trusted guests, as it wouldn't provide any security. But it could be useful for if you wanted to run a NAS or similar off a single HBA for the host + NAS. You'd also have to be careful to not pass through disks to any guests that will happily mount everything you throw at it (Windows), since you need a specialized filesystem to support concurrent access.
Ok, but then that needs all sorts of plumbing that doesn't exist and is unlikely to exist (see the relatively poor handling of SR-IOV NICs, and those have been around for literally a decade and a half):
- The driver needs to handle all the housekeeping involved in this
- The host needs an interface to tell the HBA where to assign disks
- The hypervisor needs to query HBA VFs
- The hypervisor needs to use the above-mentioned interface to assign disks
- etc...
I'd consider that a complete non-starter. Akin to staring down the barrel of loaded rifle while your drunk friend has the finger on the trigger while balancing on one leg.
You may very likely even encounter at least one thread from myself in this very forum about SR-IOV on SAS Controllers, and the conclusion is that there is barely any proof about anyone getting it working, nor explaining how it works, not how to configure it. Yet SR-IOV itself was on the product sheet of many LSI controllers. Go figure...
You do absolutely NOTHING with that alone. The card drivers have to actually enable the Virtual Functions. Plus they have to... do something when you pass them. No one knowns how the whole piping is supposed to work.
Want another example? Intel Tiger Lake, Alder Lake and Raptor Lake IGPs have SR-IOV exposed, but you can do nothing with that alone. With the difference being that drivers for those exists, and they have yet to be upstreamed, but work.
No i dont. The point has been stated a half dozen times. The driver has no usable exposure. Probably even cards newer than 9200 series have no firmware developed for this. And even 9200 series does not seem to be available.
It was mentioned in a white paper. A demo was shown. Then it disappeared. Welcome to technology.