Hi,
I have a Proxmox cluster with CEPH. It contains 5 servers with ConnectX-2 adapters running CEPH using IPoIB. It runs at 40Gbit with not very good performance (large MTU 64k ?). Servers are 1U, so only 1 PCIe slot
Now I want to replace ConnectX-2 (IPoIB) with ConnectX-3 adapters running in ETH mode at 40 Gbit (MTU 1500).
I plan to do this on the fly - without stopping the whole cluster - one server at a time...
BUT, I have a problem that the old adapters run in IPoIB mode, but the new ones will run in ETH.
Is it possible to run one network on IBoIP + ETH on one SX6036 switch?
Is it possible to connect IPoIB and ETH adapters with one SX6036?
Now I have 1 machine connected to switch using ETH, the others are IPoIB and can't see ETH <-> IPoIB.
My SX6036 has the following licenses:
- GW ports number 1 (ok)
- IB enabled true (ok)
- IB port SW HDR speed limitation (ok)
- Eth enabled true (ok)
- Eth port SW speed limit 100Gb (ok)
- Eth L2 enabled true (ok)
- Eth L3 enabled true (ok)
- FCF enabled true (ok)
Thanks for your advice
I have a Proxmox cluster with CEPH. It contains 5 servers with ConnectX-2 adapters running CEPH using IPoIB. It runs at 40Gbit with not very good performance (large MTU 64k ?). Servers are 1U, so only 1 PCIe slot
Now I want to replace ConnectX-2 (IPoIB) with ConnectX-3 adapters running in ETH mode at 40 Gbit (MTU 1500).
I plan to do this on the fly - without stopping the whole cluster - one server at a time...
BUT, I have a problem that the old adapters run in IPoIB mode, but the new ones will run in ETH.
Is it possible to run one network on IBoIP + ETH on one SX6036 switch?
Is it possible to connect IPoIB and ETH adapters with one SX6036?
Now I have 1 machine connected to switch using ETH, the others are IPoIB and can't see ETH <-> IPoIB.
My SX6036 has the following licenses:
- GW ports number 1 (ok)
- IB enabled true (ok)
- IB port SW HDR speed limitation (ok)
- Eth enabled true (ok)
- Eth port SW speed limit 100Gb (ok)
- Eth L2 enabled true (ok)
- Eth L3 enabled true (ok)
- FCF enabled true (ok)
Thanks for your advice
Last edited: