Hello,
I have 3 servers (actually a few more, but they are connected over LAN segment) and 1 switch 7050QX-32S (as SAN switch)
The servers are used for following:
1st - Ubuntu 22.04 (6.5) + 6xPM9A3 (1.92TB) + SPDK (24.01) to provide NVMe-oF target.
2nd and 3rd are ESXi 7.0U3
All hosts are using ConnectX-3 Pro with RoCEv2 enabled and connected through 7050QX-32S
Right now I'm trying to setup lossless ethernet (DCB).
For ESXi I have found reference for connectX-4, which I have modified for ConnectX-3 Pro like this
And from switch side I have followed mellanox guide for Arista, except for DCBX LLDP, since I don't know how to enable it from ESXi side
Port config:
Now I don't know how to configure the same PCP, DSCP and QoS from Ubuntu side. May someone help me?
I have 3 servers (actually a few more, but they are connected over LAN segment) and 1 switch 7050QX-32S (as SAN switch)
The servers are used for following:
1st - Ubuntu 22.04 (6.5) + 6xPM9A3 (1.92TB) + SPDK (24.01) to provide NVMe-oF target.
2nd and 3rd are ESXi 7.0U3
All hosts are using ConnectX-3 Pro with RoCEv2 enabled and connected through 7050QX-32S
Right now I'm trying to setup lossless ethernet (DCB).
For ESXi I have found reference for connectX-4, which I have modified for ConnectX-3 Pro like this
Code:
esxcli system module parameters set -m nmlx4_en -p "pfctx=0x08 pfcrx=0x08"
esxcli system module parameters set -m nmlx4_rdma -p "dscp_force=26 pcp_force=3"
esxcli system module parameters set -m nmlx4_core -p "enable_qos=1 enable_rocev2=1"
Code:
platform trident mmu queue profile RoCELosslessProfile
ingress threshold 1/16
egress unicast queue 3 threshold 8
platform trident mmu queue profile RoCELosslessProfile apply
Code:
speed forced 40gfull
qos trust dscp
priority-flow-control on
priority-flow-control priority 3 no-drop
!
tx-queue 3
bandwidth guaranteed 20000000
random-detect ecn minimum-threshold 150 kbytes maximum-threshold 1500 kbytes max-mark-probability 100 weight 0
Now I don't know how to configure the same PCP, DSCP and QoS from Ubuntu side. May someone help me?
Last edited: