I have a little issue while testing InfiniBand connections. While system (Bios-wise) is set to performance and the CPU governance is set to performance, I still receive the message/alert/warning of CPU frequency mismatches. Both CPU's are E5-2673v3's. The cards are old 10EUR eBay findings of SUN Falcons (aka Mellanox MHRH2A-XSR's). Firmware is the last 2.11.2010 and I'm unable to get RDMA working.
Here is a test result with the alert/message/warning:
Ok, I can drop the message with -F, but even aligning the CPU's to performance mode did not help or just lowered the gap of the CPU frequencies. Is there any more room for optimization in this case?
Thank you in advance.
Here is a test result with the alert/message/warning:
Code:
root@pve2:~# ib_send_lat 192.168.1.71
---------------------------------------------------------------------------------------
Send Latency Test
Dual-port : OFF Device : mlx4_0
Number of qps : 1 Transport type : IB
Connection type : RC Using SRQ : OFF
TX depth : 1
Mtu : 4096[B]
Link type : IB
Max inline data : 236[B]
rdma_cm QPs : OFF
Data ex. method : Ethernet
---------------------------------------------------------------------------------------
local address: LID 0x01 QPN 0x0211 PSN 0x5fe420
remote address: LID 0x02 QPN 0x0211 PSN 0x2d3b1c
---------------------------------------------------------------------------------------
#bytes #iterations t_min[usec] t_max[usec] t_typical[usec] t_avg[usec] t_stdev[usec] 99% percentile[usec] 99.9% percentile[usec]
Conflicting CPU frequency values detected: 3073.762000 != 2856.115000. CPU Frequency is not max.
Conflicting CPU frequency values detected: 3040.750000 != 2727.213000. CPU Frequency is not max.
2 1000 0.74 9.07 0.77 0.81 0.31 1.70 9.07
---------------------------------------------------------------------------------------
root@pve2:~# ib_send_bw 192.168.1.71
---------------------------------------------------------------------------------------
Send BW Test
Dual-port : OFF Device : mlx4_0
Number of qps : 1 Transport type : IB
Connection type : RC Using SRQ : OFF
TX depth : 128
CQ Moderation : 100
Mtu : 4096[B]
Link type : IB
Max inline data : 0[B]
rdma_cm QPs : OFF
Data ex. method : Ethernet
---------------------------------------------------------------------------------------
local address: LID 0x01 QPN 0x0212 PSN 0x7dca80
remote address: LID 0x02 QPN 0x0212 PSN 0xc1ba80
---------------------------------------------------------------------------------------
#bytes #iterations BW peak[MB/sec] BW average[MB/sec] MsgRate[Mpps]
Conflicting CPU frequency values detected: 2967.297000 != 2432.842000. CPU Frequency is not max.
65536 1000 3756.01 3755.94 0.052095
---------------------------------------------------------------------------------------
root@pve2:~#
Thank you in advance.