SR-IOV of PM1733 does not seem to work.

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Da_iaji

New Member
Aug 11, 2021
10
0
1
I use the 5.13.9 version of the kernel, and SR-IOV support has been enabled in the BIOS.
Code:
echo 4 > /sys/class/nvme/nvme0/device/sriov_numvfs

dmesg | grep -E 'nvme|pci'

[  154.374020] pci 0000:01:00.2: [144d:a824] type 00 class 0x010802
[  154.374261] pci 0000:01:00.2: Adding to iommu group 32
[  154.374455] nvme nvme1: pci function 0000:01:00.2
[  154.374486] pci 0000:01:00.3: [144d:a824] type 00 class 0x010802
[  154.374676] pci 0000:01:00.3: Adding to iommu group 33
[  154.374696] nvme 0000:01:00.2: enabling device (0000 -> 0002)
[  154.374785] nvme nvme2: pci function 0000:01:00.3
[  154.374826] nvme 0000:01:00.3: enabling device (0000 -> 0002)
[  154.374855] pci 0000:01:00.4: [144d:a824] type 00 class 0x010802
[  154.375232] pci 0000:01:00.4: Adding to iommu group 34
[  154.375351] nvme nvme3: pci function 0000:01:00.4
[  154.375379] pci 0000:01:00.5: [144d:a824] type 00 class 0x010802
[  154.375387] nvme 0000:01:00.4: enabling device (0000 -> 0002)
[  154.375683] pci 0000:01:00.5: Adding to iommu group 35
[  154.375785] nvme nvme4: pci function 0000:01:00.5
[  154.375798] nvme 0000:01:00.5: enabling device (0000 -> 0002)
[  184.877170] nvme nvme1: Device not ready; aborting initialisation, CSTS=0x2
[  184.877171] nvme nvme4: Device not ready; aborting initialisation, CSTS=0x2
[  184.877173] nvme nvme3: Device not ready; aborting initialisation, CSTS=0x2
[  184.877176] nvme nvme1: Removing after probe failure status: -19
[  184.877177] nvme nvme4: Removing after probe failure status: -19
[  184.877179] nvme nvme3: Removing after probe failure status: -19
[  184.879156] nvme nvme2: Device not ready; aborting initialisation, CSTS=0x2
[  184.879160] nvme nvme2: Removing after probe failure status: -19
Is this a similar bug?

I tested other PM1733 and motherboards, and they seem to have similar faults.
 
Last edited:

MrCalvin

IT consultant, Denmark
Aug 22, 2016
87
15
8
51
Denmark
www.wit.dk
I assume this if for paththrough in a guest or is it for general performance improvement on the host?

Did you see this work with other disks than the Samsung PM1733?

Just curious on subject :)
 

Da_iaji

New Member
Aug 11, 2021
10
0
1
I assume this if for paththrough in a guest or is it for general performance improvement on the host?

Did you see this work with other disks than the Samsung PM1733?

Just curious on subject :)
There is no doubt that SSDs that support SR-IOV are almost all high-end products even in data center-level hard drives. So in fact, as far as I know, SR-IOV is only supported by PM1733 PM1735 and the older PM1725. Unfortunately, this I got the block PM1733 from the miner at a very cheap price, and I have never owned his hard drive that supports SR-IOV.

As for the usage scenarios, both are. Obviously, as a flagship SSD, the performance is already the first echelon (excluding SLC and Optane SSDs). There is no doubt that it can speed up my HOST. After SR-IOV is enabled, it can Through PCI pass-through, my VM instance also gets I/O performance improvement.
 

RTM

Well-Known Member
Jan 26, 2014
956
359
63
Have you made sure to enable IOMMU and all that stuff in your OS?

I have not used SR-IOV but I assume the process should be similar to what you do with a NIC, so perhaps you can follow this tutorial (for Proxmox):

At least the IOMMU part should be similar (unless you don't use something based on Debian).
 

Da_iaji

New Member
Aug 11, 2021
10
0
1
Have you made sure to enable IOMMU and all that stuff in your OS?

I have not used SR-IOV but I assume the process should be similar to what you do with a NIC, so perhaps you can follow this tutorial (for Proxmox):

At least the IOMMU part should be similar (unless you don't use something based on Debian).
I have enabled IOMMU in the BIOS.
 

Da_iaji

New Member
Aug 11, 2021
10
0
1
You need to enable it in the OS as well.

That tutorial I linked to the Proxmox forum, mentions that you want to enable ACS in BIOS too.
ACS has also been enabled.
I have passed the GPU straight.
 

RTM

Well-Known Member
Jan 26, 2014
956
359
63
I have passed the GPU straight.
I assume you mean that you are able to passthrough a GPU to a VM?
In that case IOMMU should be enabled (and working).

On another note, I found the following post on stackoverflow, it deals with the same disk that you are using, apparently you have to allocate resources on the disk to use SR-IOV:
Have you seen it already? Look at the accepted post from the user "Umamahesh Allenki"
 

Da_iaji

New Member
Aug 11, 2021
10
0
1
I assume you mean that you are able to passthrough a GPU to a VM?
In that case IOMMU should be enabled (and working).

On another note, I found the following post on stackoverflow, it deals with the same disk that you are using, apparently you have to allocate resources on the disk to use SR-IOV:
Have you seen it already? Look at the accepted post from the user "Umamahesh Allenki"
I tried virt-mgmt, but it doesn't seem to work.