I don't fix anything. I just flip some switches in the BIOS. Newer BIOS's might fix this, but the problem is they messed with things like C-states that you can't undo, making them not feasible to use.
CWWK i5-1235U 6 port i226 report
- Thread starter fta
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Has anyone successfully used Proxmox USB passthrough on this hardware? I keep getting guest VM crashes while transferring data from a USB 3.0 drive passed through either by port or by device ID about 15 seconds into the transfer.
According to this forum, the crash is likely because the USB controller is in the same IOMMU group as another essential device (in my case RAM)
I've tried all external USB ports (4x USB-A and USB-C) but all are on the same USB controller and I don't see any USB headers on the motherboard.
If this is correct, and there's no way to modify the IOMMUs, the only thing I can think is to use the secondary m.2 slot to add a USB controller like this. Seems really janky though.
According to this forum, the crash is likely because the USB controller is in the same IOMMU group as another essential device (in my case RAM)
Bash:
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done
...
IOMMU group 5 00:0d.0 USB controller [0c03]: Intel Corporation Alder Lake-P Thunderbolt 4 USB Controller [8086:461e] (rev 04)
IOMMU group 6 00:14.0 USB controller [0c03]: Intel Corporation Alder Lake PCH USB 3.2 xHCI Host Controller [8086:51ed] (rev 01)
IOMMU group 6 00:14.2 RAM memory [0500]: Intel Corporation Alder Lake PCH Shared SRAM [8086:51ef] (rev 01)If this is correct, and there's no way to modify the IOMMUs, the only thing I can think is to use the secondary m.2 slot to add a USB controller like this. Seems really janky though.
Last edited:
i am passing through an usb bluetooth adapter to my home assistant vm in proxmox.
my iommu groups look like this:
My USB passthrough Config looks like this:
alternatively i could also pass through the whole USB-port:
my /etc/kernel/cmdline looks like this:
my iommu groups look like this:
Code:
root@pve:~# for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done
IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation Alder Lake-UP3 GT1 [UHD Graphics] [8086:46b3] (rev 0c)
IOMMU group 10 00:17.0 SATA controller [0106]: Intel Corporation Alder Lake-P SATA AHCI Controller [8086:51d3] (rev 01)
IOMMU group 11 00:19.0 Serial bus controller [0c80]: Intel Corporation Alder Lake-P Serial IO I2C Controller #0 [8086:51c5] (rev 01)
IOMMU group 11 00:19.1 Serial bus controller [0c80]: Intel Corporation Alder Lake-P Serial IO I2C Controller #1 [8086:51c6] (rev 01)
IOMMU group 12 00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:51bc] (rev 01)
IOMMU group 13 00:1c.5 PCI bridge [0604]: Intel Corporation Device [8086:51bd] (rev 01)
IOMMU group 14 00:1c.6 PCI bridge [0604]: Intel Corporation Device [8086:51be] (rev 01)
IOMMU group 15 00:1c.7 PCI bridge [0604]: Intel Corporation Alder Lake PCH-P PCI Express Root Port #9 [8086:51bf] (rev 01)
IOMMU group 16 00:1d.0 PCI bridge [0604]: Intel Corporation Alder Lake PCI Express Root Port [8086:51b0] (rev 01)
IOMMU group 17 00:1d.1 PCI bridge [0604]: Intel Corporation Alder Lake PCI Express x1 Root Port #10 [8086:51b1] (rev 01)
IOMMU group 18 00:1f.0 ISA bridge [0601]: Intel Corporation Alder Lake PCH eSPI Controller [8086:5182] (rev 01)
IOMMU group 18 00:1f.4 SMBus [0c05]: Intel Corporation Alder Lake PCH-P SMBus Host Controller [8086:51a3] (rev 01)
IOMMU group 18 00:1f.5 Serial bus controller [0c80]: Intel Corporation Alder Lake-P PCH SPI Controller [8086:51a4] (rev 01)
IOMMU group 19 01:00.0 Non-Volatile memory controller [0108]: MAXIO Technology (Hangzhou) Ltd. NVMe SSD Controller MAP1202 [1e4b:1202] (rev 01)
IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:4619] (rev 04)
IOMMU group 20 02:00.0 Network controller [0280]: Intel Corporation Device [8086:272b] (rev 1a)
IOMMU group 21 03:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU group 22 04:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU group 23 05:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU group 24 06:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU group 25 07:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU group 26 08:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU group 2 00:04.0 Signal processing controller [1180]: Intel Corporation Alder Lake Innovation Platform Framework Processor Participant [8086:461d] (rev 04)
IOMMU group 3 00:06.0 PCI bridge [0604]: Intel Corporation 12th Gen Core Processor PCI Express x4 Controller #0 [8086:464d] (rev 04)
IOMMU group 4 00:06.2 PCI bridge [0604]: Intel Corporation 12th Gen Core Processor PCI Express x4 Controller #2 [8086:463d] (rev 04)
IOMMU group 5 00:0a.0 Signal processing controller [1180]: Intel Corporation Platform Monitoring Technology [8086:467d] (rev 01)
IOMMU group 6 00:0d.0 USB controller [0c03]: Intel Corporation Alder Lake-P Thunderbolt 4 USB Controller [8086:461e] (rev 04)
IOMMU group 7 00:14.0 USB controller [0c03]: Intel Corporation Alder Lake PCH USB 3.2 xHCI Host Controller [8086:51ed] (rev 01)
IOMMU group 7 00:14.2 RAM memory [0500]: Intel Corporation Alder Lake PCH Shared SRAM [8086:51ef] (rev 01)
IOMMU group 8 00:15.0 Serial bus controller [0c80]: Intel Corporation Alder Lake PCH Serial IO I2C Controller #0 [8086:51e8] (rev 01)
IOMMU group 8 00:15.1 Serial bus controller [0c80]: Intel Corporation Alder Lake PCH Serial IO I2C Controller #1 [8086:51e9] (rev 01)
IOMMU group 9 00:16.0 Communication controller [0780]: Intel Corporation Alder Lake PCH HECI Controller [8086:51e0] (rev 01)alternatively i could also pass through the whole USB-port:
my /etc/kernel/cmdline looks like this:
Code:
root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt split_lock_detect=off
Last edited:
Thanks. I see your USB controller is also sharing the IOMMU group as the RAM, so it sounds like it's baked into hardware/BIOS.
I have intel_iommu=on but not iommu=pt nor split_lock_detect=off so I'll try those.
I'm now wondering if it only crashes under high USB resource utilization like file transfers and not BT/Zigbee etc. I'll find out when I move my Home Assistant+zigbee adapter to a VM on this host later.
Update: iommu=pt and split_lock_detect=off didn't fix the issue. I also notice that the transfer starts at 150MB/s but slows linearly to around 30MB/s before crashing.
Last edited:
That would be awesome. FYI, my VM is a Mint 21.3 running the latest qemu-guest-agent, but if the problem is indeed the shared IOMMU then it may occur with other VM host OSs as well.
i Just passed the usb ssd to a debian 12 vm and ran a dd of 100GB from /dev/random to it.
speed isnt great since it is NTFS formatted, the vm has only 2 cpu's and /dev/random isnt awesome as a source, but nothing happened to the vm.
this is how the passthrough looks:
anything else i should test?
i can try to install mint. maybe its something mint-specific, or specific to the usb-drive you are using.
mine is a noname chinese usb to sata cable with a crucial mx100 attached to it.
speed isnt great since it is NTFS formatted, the vm has only 2 cpu's and /dev/random isnt awesome as a source, but nothing happened to the vm.
this is how the passthrough looks:
anything else i should test?
i can try to install mint. maybe its something mint-specific, or specific to the usb-drive you are using.
mine is a noname chinese usb to sata cable with a crucial mx100 attached to it.
i will install mint next and use the usb ssd as boot drive for the vm. lets see how it handles that.
oh by the way what proxmox version are you running? im on 8.1 with the latest updates.
edit: installed mint on the usb drive inside the vm and ran the same test again (except this time with /dev/zero as source to stress the drive more and not the cpu).
no crashes. all seems normal.
oh by the way what proxmox version are you running? im on 8.1 with the latest updates.
edit: installed mint on the usb drive inside the vm and ran the same test again (except this time with /dev/zero as source to stress the drive more and not the cpu).
no crashes. all seems normal.
Last edited:
That’s correct. What I have linked in the post are what I recommend. The first BIOS we had access to is the only one they didn’t mess with.
Thanks for all your help.
I ran more tests and I'm quite certain the shared IOMMU is the problem.
- Issue is reproducible on Mint, Debian 12, and Ubuntu 23.10 guests.
- Windows 11 guest doesn't crash but transfers are ridiculously slow (177KB/s)
- Issue occurs writing to HDDs in two different enclosures (jms578 chip and jms580 chip) as well as a Sandisk flashdrive. I was unable to reproduce the issue writing to a Samsung USB 3.1 SSD but only tested once.
- Reading from all devices to the guest filesystem is fine
- The crash is consistent when trying to transfer a large folder with many files (2000+ files 20GB) but smaller writes work so far
- I tried to simulate this with the Gnome disks benchmark tool writing 50MiB * 300 cycles (15GB) but it didn't crash
- Gnome disks also reports no disk issues nor filesystem issues on all drives
- The PVE syslog includes the following error just before the crash - pve QEMU[337003]: kvm: ../hw/usb/core.c:563: usb_packet_check_state: Assertion `!"usb packet state check failed"' failed.
Code:Feb 18 20:17:01 pve CRON[342009]: pam_unix(cron:session): session closed for user rootFeb 18 20:18:16 pve kernel: usb 4-3: reset SuperSpeed Plus Gen 2x1 USB device number 2 using xhci_hcd Feb 18 20:18:21 pve kernel: usb 4-4.2: reset SuperSpeed USB device number 4 using xhci_hcd Feb 18 20:18:53 pve kernel: usb 4-4.2: reset SuperSpeed USB device number 4 using xhci_hcd Feb 18 20:18:53 pve QEMU[337003]: kvm: ../hw/usb/core.c:563: usb_packet_check_state: Assertion `!"usb packet state check failed"' failed. Feb 18 20:18:53 pve pvedaemon[324711]: <root@pam> end task UPID:pve:000524B1:0077D692:65D1E2E9:vncproxy:109:root@pam: OK Feb 18 20:18:53 pve kernel: fwbr109i0: port 2(tap109i0) entered disabled state Feb 18 20:18:53 pve kernel: tap109i0 (unregistering): left allmulticast mode Feb 18 20:18:53 pve kernel: fwbr109i0: port 2(tap109i0) entered disabled state Feb 18 20:18:53 pve systemd[1]: 109.scope: Deactivated successfully. Feb 18 20:18:53 pve systemd[1]: 109.scope: Consumed 9min 51.445s CPU time. Feb 18 20:18:53 pve qmeventd[342570]: Starting cleanup for 109 Feb 18 20:18:53 pve kernel: fwbr109i0: port 1(fwln109i0) entered disabled state Feb 18 20:18:53 pve kernel: vmbr0: port 6(fwpr109p0) entered disabled state Feb 18 20:18:53 pve kernel: fwln109i0 (unregistering): left allmulticast mode Feb 18 20:18:53 pve kernel: fwln109i0 (unregistering): left promiscuous mode Feb 18 20:18:53 pve kernel: fwbr109i0: port 1(fwln109i0) entered disabled state Feb 18 20:18:53 pve kernel: fwpr109p0 (unregistering): left allmulticast mode Feb 18 20:18:53 pve kernel: fwpr109p0 (unregistering): left promiscuous mode Feb 18 20:18:53 pve kernel: vmbr0: port 6(fwpr109p0) entered disabled state Feb 18 20:18:53 pve qmeventd[342570]: Finished cleanup for 109
Last edited:
I finalized my build and I must say that I'm really satisfied with this machine. I have the 8505 version and I'm using it as switch/NAS/media server/download box, running on Arch linux. I have two NVME drives and two SATA drives and after some tweaking it uses less than 8W in idle with one active network adapter. I put 120mm fan running at 700rpm (practically inaudible) on top of it using double sided tape and the CPU temperatures are around 20C in idle (which is quite unrealistic, since room temperature is 23C 
) and never exceeds 65C under sustained full load (package power around 35W).
One SATA drive is attached on the inside and second on the outside of the lid (again with the double sided tape). The NVME drive on the NVME extension adapter is running at 1x PCIe 4.0 and it can achieve around 1.7GB/s read and write speeds, which is not bad.
I'm using original Dajing 65W power adapter, I tried it also with delta and lite-on adapters, but both of them had idle power consumption higher by 2W for some reason, so the Dajing is actually the most efficient.
The idle power consumption also depends on used SATA drives, now I'm using 4TB Crucial mx500 and 512GB Samsung 850 evo, but when I used Verbatim VI550 the power consumption was 2W higher. I'm planning to replace the Samsung drive with another 4TB Cucial in the future (maybe when the prices will drop again).
Many thanks to @fta for the unlocked BIOS.
) and never exceeds 65C under sustained full load (package power around 35W).One SATA drive is attached on the inside and second on the outside of the lid (again with the double sided tape). The NVME drive on the NVME extension adapter is running at 1x PCIe 4.0 and it can achieve around 1.7GB/s read and write speeds, which is not bad.
I'm using original Dajing 65W power adapter, I tried it also with delta and lite-on adapters, but both of them had idle power consumption higher by 2W for some reason, so the Dajing is actually the most efficient.
The idle power consumption also depends on used SATA drives, now I'm using 4TB Crucial mx500 and 512GB Samsung 850 evo, but when I used Verbatim VI550 the power consumption was 2W higher. I'm planning to replace the Samsung drive with another 4TB Cucial in the future (maybe when the prices will drop again).
Many thanks to @fta for the unlocked BIOS.
Attachments
How did you fix the TOP fan into the chassis?I finalized my build and I must say that I'm really satisfied with this machine. I have the 8505 version and I'm using it as switch/NAS/media server/download box, running on Arch linux. I have two NVME drives and two SATA drives and after some tweaking it uses less than 8W in idle with one active network adapter. I put 120mm fan running at 700rpm (practically inaudible) on top of it using double sided tape and the CPU temperatures are around 20C in idle (which is quite unrealistic, since room temperature is 23C
One SATA drive is attached on the inside and second on the outside of the lid (again with the double sided tape). The NVME drive on the NVME extension adapter is running at 1x PCIe 4.0 and it can achieve around 1.7GB/s read and write speeds, which is not bad.
I'm using original Dajing 65W power adapter, I tried it also with delta and lite-on adapters, but both of them had idle power consumption higher by 2W for some reason, so the Dajing is actually the most efficient.
The idle power consumption also depends on used SATA drives, now I'm using 4TB Crucial mx500 and 512GB Samsung 850 evo, but when I used Verbatim VI550 the power consumption was 2W higher. I'm planning to replace the Samsung drive with another 4TB Cucial in the future (maybe when the prices will drop again).
Many thanks to @fta for the unlocked BIOS.
I'm using those Noctua Rubbers, but they don't inspire too much reliability xD
Double-sided tape in the corners, it holds very firmly
Attachments
ok, you're a genius. I should have done that.
Lemme send you what I've done.
I mean, it works.... LOL.
Hi, i just did the BIOS ppdate from post #4 and i think i bricked my system. The BIOS ppdate run through, the PC rebooted and now i got a black screen and left it like that for a long time. Since nothing happened i turned off the PC and tried a CMOS reset but still get the black screen any further ideas or did i just produce an expensive brick ?
?