Hello all,
Long-time lurker, first time poster. I was unsure whether this would be more suitable in the Networking or Solaris section but settled over here. To the point, I am trying to configure my Infiniband cards between my Solaris ZFS storage appliance and my ESXi 5.5 host. I started on the storage side to try and get the card to work. Specifically the cards are Voltaire HCA 500Ex-D HCA-00001 units. Dual port DDR 4x Infiniband gear. At first, there was no reference to the card, but I noticed that under prtconf, the system listed that there was no driver attached to the card. After adding the ib packages (basically anything with the label infiband or ib in the package repository, which included the Mellanox ConnectX driver), I now see the card under prtconf
A paste of prtconf -v, limited to only the pci instance of the card.
Namely, it mentions that it how now got the proper hermon0 driver attached (which should be default now that Tavor is gone in 11.1). The problem is that cfgadm is completely empty. Shouldn't there be an HCA listed? There are currently no lights on any of the ports of my IB card. The host and storage device are directly attached via CX4. No IB switch is involved (I intend to run OpenSM off the ESXi host). I have not worked on the ESXi host side of things yet.
Is there an obvious step I'm missing to get a configurable HCA in cfgadm or some sort of NIC port? While ultimately I want to run SRP, I'd like to have some sort of baseline that I know works. Obviously right now it doesn't work. I appreciate any advice or clues that can be given, and of course I'm happy to pull up any more information that might help!
Thanks Community!
Long-time lurker, first time poster. I was unsure whether this would be more suitable in the Networking or Solaris section but settled over here. To the point, I am trying to configure my Infiniband cards between my Solaris ZFS storage appliance and my ESXi 5.5 host. I started on the storage side to try and get the card to work. Specifically the cards are Voltaire HCA 500Ex-D HCA-00001 units. Dual port DDR 4x Infiniband gear. At first, there was no reference to the card, but I noticed that under prtconf, the system listed that there was no driver attached to the card. After adding the ib packages (basically anything with the label infiband or ib in the package repository, which included the Mellanox ConnectX driver), I now see the card under prtconf
A paste of prtconf -v, limited to only the pci instance of the card.
Code:
pci15b3,634a, instance #0
System software properties:
name='ddi-forceattach' type=int items=1
value=00000001
name='active-dma-flush' type=int items=1
value=00000001
name='ddi-vhci-class' type=string items=1
value='ib'
name='iommu-dvma-mode' type=string items=1
value='unity'
Driver properties:
name='fm-accchk-capable' type=boolean dev=none
name='fm-ereport-capable' type=boolean dev=none
Hardware properties:
name='assigned-addresses' type=int items=10
value=83020010.00000000.fbd00000.00000000.00100000.c3020018.00000000.fa000000.00000000.00800000
name='reg' type=int items=15
value=00020000.00000000.00000000.00000000.00000000.03020010.00000000.00000000.00000000.00100000.43020018.00000000.00000000.00000000.00800000
name='compatible' type=string items=13
value='pciex15b3,634a.15b3.634a.a0' + 'pciex15b3,634a.15b3.634a' + 'pciex15b3,634a.a0' + 'pciex15b3,634a' + 'pciexclass,0c0600' + 'pciexclass,0c06' + 'pci15b3,634a.15b3.634a.a0' + 'pci15b3,634a.15b3.634a' + 'pci15b3,634a' + 'pci15b3,634a.a0' + 'pci15b3,634a' + 'pciclass,0c0600' + 'pciclass,0c06'
name='model' type=string items=1
value='InfiniBand'
name='power-consumption' type=int items=2
value=00000001.00000001
name='devsel-speed' type=int items=1
value=00000000
name='interrupts' type=int items=1
value=00000001
name='subsystem-vendor-id' type=int items=1
value=000015b3
name='subsystem-id' type=int items=1
value=0000634a
name='unit-address' type=string items=1
value='0'
name='class-code' type=int items=1
value=000c0600
name='revision-id' type=int items=1
value=000000a0
name='vendor-id' type=int items=1
value=000015b3
name='device-id' type=int items=1
value=0000634a
Device Minor Nodes:
dev=(307,0)
dev_path=/pci@0,0/pci8086,3c04@2/pci15b3,634a@0:devctl
spectype=chr type=minor
dev_link=/dev/infiniband/hca/hermon0Is there an obvious step I'm missing to get a configurable HCA in cfgadm or some sort of NIC port? While ultimately I want to run SRP, I'd like to have some sort of baseline that I know works. Obviously right now it doesn't work. I appreciate any advice or clues that can be given, and of course I'm happy to pull up any more information that might help!
Thanks Community!
