Infiniband ESXi Storage Node

shindo

New Member
May 7, 2013
26
0
1
WI, USA
I am attempting to rebuild (again) a storage node for my C6100 and have a couple of questions. Here is my proposed setup:

  • 6 disks on onboard sata
  • 6 disks on Areca ARC-1222
  • Mellanox dual port QDR Infiniband Daughter Card - JR3P1 directly connected to two other nodes with the same hardware
  • ESXi on USB disk
  • OmniOS & Napp-it VM to present storage to other ESXi nodes
  • CentOS VM for OpenSM

I have been configuring the disks on the Areca as passthrough (Areca's passthrough, as opposed to RAID or JBOD) and using one for the initial datastore for the VMs.

Here are my questions/issues:

  1. Concerning the Infiniband card, do I need to configure this as pass-through HBAs within ESXi or build a virtual switch using it? I am hoping to use SRP, if that matters.
  2. I have installed the drivers from Areca's website for the ARC-1222, but I can only see one of the disks on it (currently 3 connected). I am able to create a datastore on this disk. The details of the hba shows: Targets: 1, Devices: 1, Paths:2. It then shows one path to C0:T0L0 LUN 0 as Active (I/O) and C0:T0:L1 Lun 1 as Dead. Also, ESXi is recognizing the card as an ARC-1680. Any ideas on what I can do to get the other connected disks to be recognized? The web interface on the Areca shows all three disks in a Normal state.

Thanks for your help. I'm learning a lot with this project, but man have I hit a lot of roadblocks!
 

shindo

New Member
May 7, 2013
26
0
1
WI, USA
Update:

If I remove all of the pass-through disks in the Areca utility and then recreate them, ESXi seems to be able to pick up only the first disk I set as pass-through. It doesn't matter which disk it is, it just recognizes the first one I configure and neither of the other two. I also tried switching the card to JBOD mode to see what would happen, but ESXi didn't recognize any disks at that point.
 

jsummers

New Member
Mar 4, 2013
3
0
0
Shindo,

If you are looking for help on this I would recommend the Mellanox Community it is open community with tons of experts on hand and I am sure they will be able to help you out.