I recently purchased 3 HP 649281-B21:
HP 656089-001 Infiniband 10/40GB DP NIC 649281-B21+ QSFP /SFP's | eBay
649281-B21 656089-001 661685-001 HP IB FDR/EN 10/40GB 2P 544QSFP ADAPTER
I am connecting all 3 together with 3 cables:
Amazon.com: QSFP+ to QSFP+ 40Gb Passive DAC Copper Infiniband Cable Direct Attach QSFP 31 inch 80cm: Computers & Accessories
Two nodes are HP DL 380 G7s running ESXi. They fired right up, and got link between each other.
The other node is a custom built FreeNas machine. Each ESXi box is connected to the FreeNas machine. However, I am unable to get link between either ESXi node and the FreeNas node. FreeNas does recognize the NIC:
[root@filejunky] ~# ifconfig mlxen0
mlxen0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:02:c9:fb:a3:30
inet 10.3.3.1 netmask 0xffff0000 broadcast 10.3.255.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect
status: no carrier
[root@filejunky] ~# ifconfig mlxen1
mlxen1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:02:c9:fb:a3:31
inet 10.4.3.1 netmask 0xffff0000 broadcast 10.4.255.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect
status: no carrier
This NIC is the "odd man out". The other two were purchased as a pair, this one was not.
Any thoughts on getting link between the FreeNas machine and ESXi?
HP 656089-001 Infiniband 10/40GB DP NIC 649281-B21+ QSFP /SFP's | eBay
649281-B21 656089-001 661685-001 HP IB FDR/EN 10/40GB 2P 544QSFP ADAPTER
I am connecting all 3 together with 3 cables:
Amazon.com: QSFP+ to QSFP+ 40Gb Passive DAC Copper Infiniband Cable Direct Attach QSFP 31 inch 80cm: Computers & Accessories
Two nodes are HP DL 380 G7s running ESXi. They fired right up, and got link between each other.
The other node is a custom built FreeNas machine. Each ESXi box is connected to the FreeNas machine. However, I am unable to get link between either ESXi node and the FreeNas node. FreeNas does recognize the NIC:
[root@filejunky] ~# ifconfig mlxen0
mlxen0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:02:c9:fb:a3:30
inet 10.3.3.1 netmask 0xffff0000 broadcast 10.3.255.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect
status: no carrier
[root@filejunky] ~# ifconfig mlxen1
mlxen1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:02:c9:fb:a3:31
inet 10.4.3.1 netmask 0xffff0000 broadcast 10.4.255.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect
status: no carrier
This NIC is the "odd man out". The other two were purchased as a pair, this one was not.
Any thoughts on getting link between the FreeNas machine and ESXi?