Search results

Trying to contribute a data-point from side-band. As part of an effort to debug a NVME (SN570 1TB) pass-through problem (FLR causing AER / DPC error related to link low power state), I attempted to force my NVME out of the ASPM L1 mode. This bumps the machine power by ~3W at the outlet. PS...

BTW, it might be worthy to name those cards that do not work with ASPM according to your experiment. Other people who care may benefit from the list. And other card owners who had different experience can also chime-in...

The link from @glow suggests that CX4 may be able to work with ASPM enabled. So probably newer cards have higher chance to work... I also remember the Intel x710 datasheet describing ASPM functionality, however I haven't seen any real world evidence from kernel log...

It's true that the motherboard is playing a major role here. I just realize that my Asrock consumer board forcefully disable this ASPM feature through FADT. This even defeat the pcie_aspm=force Linux kernel command-line: $ dmesg|grep ASPM [ 0.030562] PCIe ASPM is forcibly enabled [...

@Mithril, thanks for the help! Looking forward to your result!

I tried some dirty trick from this post to manually force enable the ASPM L0S mode. Unfortunately this lead to floods of error message in the kernel log: I suspect the claim on ASPM support from this CX3 could be fake or at least buggy ... Given the end-of-life status of this card I think we...

The saving on switch is definite. However the saving from 'the port itself powers completely down when the client is switched off' could be marginal. I think we really need a good card with good driver to approach this part of the saving...

I agree with you. But anyway the measured number is still very disappointing to me. Note that the number I reported is in a testbench setup without any cable plugged. It's absolutely wasting power for nothing... Once I read a suggestion from FreeNAS forum saying that having 10GE connection...

Thanks for the information, @glow. Unfortunately the price of CX4 is beyond my budget :-( For X520 I could see contradicting information: According to the 82599 datasheet, it should have support to PCIE link state. But there are other information indicating ASPM related bugs here and Errata 37...

I just realize that the TDP rating is not the only important factor when looking for low power cards. It's also important to make sure that the card does not prevent the system from entering deep power saving state. I run into a situation that my CX3 card prevents my CPU to enter PC8 package...

I have a Mellanox CX312A card which I grabbed for reasonably low power spec 3.8W typical && 4.55W max (Page 48 of the manual). However, the measured power consumption change before and after the card installation is rather huge: ~10W without vs ~17.5 with the NIC. So this is a ~7.5W power tax at...

I would rather avoid the vendor driver mess if the stock one worked for me... I wasn't looking for any advanced feature from vendor driver either, just for bug fixes. I know very little about RDMA, but if it requires any client-side cooperation it won't be helpful to me... Thankfully my driver...

Thanks for the reminder, Stephan. Actually I came across that post before but I didn't realize that this may have anything to do with my issue. I didn't try any window guest yet and thus didn't run into any explicit error code. My issue is that the NIC just eat my packets in silent when the...

I think I run into a bug that with Linux stock driver (mlx4_en 4.0.0) does not handle promiscuous mode well in SRIOV PF ports. Switching to the vendor driver (v4.9-4.1.7) immediately solved my problem. I really hate this driver policy of releasing old && buggy driver to public while keep the...

Just to answer my own question. Surprisingly the vendor stock driver does make a huge difference. The driver come with the Linux kernel is stuck at version V4.0.0, and the latest version I can get from the vendor is on V4.9. Switching to the vendor driver magically solve my problem. This gives...

Being a freshman on SR-IOV config, I'm desperate and would like to hear some advice from experts. What I'm trying to do is to setup a 10GE direct link, without involving switches, between my desktop and my home server. The intended usage is to split LAN and Internet access from my desktop...