Search results

  1. J

    6.7U1 Vsan doesn't seem to work with Connectx-3

    All are HP DL580 G7's with CX354A cards in them. It looks like when ESXI recognizes the network driver it shuts down vSAN services on the server which prevent vSAN to do anything, thus blocking server from exiting maintenance mode. I'm sure its just some setting or option or oversight since...
  2. J

    6.7U1 Vsan doesn't seem to work with Connectx-3

    I give up and rolling back to 6.7. I'm unable to use MST in 6.7U1 as getting "-E- nmlx core must be loaded before starting mst." error even though its loaded.
  3. J

    6.7U1 Vsan doesn't seem to work with Connectx-3

    Thats exactly what I was getting. Now with fresh install I'm unable to add to vSAN cluster. Everything else is good just vSAN doesn't work. Also i'm not even able to add vSAN service to an 1g ethernet NIC unless i uninstall the mellanox infiniband drivers
  4. J

    6.7U1 Vsan doesn't seem to work with Connectx-3

    Does your vmkernel show vsan enabled? My bet is it doesn't which is why you cant exit maintenance mode. I had same problem. I think a couple times it showed enabled but if I removed I couldn't add it
  5. J

    6.7U1 Vsan doesn't seem to work with Connectx-3

    I've been trying to upgrade 6.7 to U1 and am unable to get vSAN service added to VMkernel. Just get general vsan error when trying. Even worse is I can get it all setup over 1G copper but when I add the drivers to the infiniband NIC and get them online it removes the vSAN service from the 1g...
  6. J

    Voltaire 4036E dropping packets

    I think i'm missing something simple here. running ConnectX-3 40GB and have a 4036E on ESXI servers and if trying to connect to my ethernet over 10GB SFP i get about 50% dropped packets when trying to ping any device over IB. I can ping the voltaire interface ip fine, the voltaire can ping...
  7. J

    Voltaire 4036 bridge ethernet

    I'm new to Infiniband, but I'm able to get IPoIB working using ESXI and the 4036e, but not sure if its passing ethernet. Do I have to do anything to pass ethernet traffic to the SFP+ interfaces?
  8. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    Looks like I was finally able to get them up using the 1.8 driver and restarting the subnet manager on the 4036e a couple times.
  9. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    Can anyone verify that MLNX-OFED-ESX-2.4.0.0-10EM-600.0.0.2494585.zip will even work with ESXI6.5? It seems like its 6.0 only. No matter what I do I can't get the link up. tried various firmware and drives and installed/uninstalled all kinds of stuff. Maybe Connectx3 IB doesn't work with...
  10. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    I haven't changed anything. I think I need to do this. HowTo Change Port Type in Mellanox ConnectX-3 A... | Mellanox Interconnect Community
  11. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    UMMMMM Not sure how to do that. I assumed they were IB originally
  12. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    Should I just get a sx6036 or sx1012? Will that provide what I'm looking for? Does it provide Ethernet. I can just put one of these cards in my router and I believe it's supported.
  13. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    If I plug it directly into another server it will work, so it seems like its the voltaire switch. Is it a setting or do I need to find a ethernet one
  14. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    yes, below is the settings, maybe something's off in there. subnet manager info is: sweep_interval: 15 max_wire_smps: 16 lmc: 0 max_op_vls: 5 transaction_timeout: 150...
  15. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    I think I understand. But i still shouldn't be getting link down in ESXI right? I think thats my main problem. Cables and everything look good and on a windows machine we're getting link just no traffic (which makes sense because nothing else on it). We tried direct attach to other card and to...
  16. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    I actually just found those posts and working through it. I'm very new at infiniband, so I cant get ethernet traffic through the IB ports? Why do they have sfp+ ethernet if i can get traffic routed to it? I'm not a network guy so its a bit over my head. also do you know what sfp+...
  17. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    this must be where i'm confused. I'm just trying to get esxi vsan on faster networking. Can I not use it for ethernet? if not can i use IB for vsan traffic? Ideally i'd like to use both ports, one for vsan and other for all else. Also I'm unable to get the sfp+ ports to work yet, think my...
  18. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    Its a 4036E which I believe is Ethernet mode, or IB but not sure. I'm not sure how to switch modes or if there's a way.
  19. J

    Mellanox Connectx-3 link down ESXI with Voltaire 4036

    hey all. I'm very new to Mellanox and IB I picked up a few MCX354A-QCBT and a Voltaire 4036 with cables. I updated the voltaire and some of the MCX354A-QCBT to the MCX354A-FCBT firmware (for 40GBE). On Windows server 2016 I show the link as up and connected but not pulling DHCP (have 1GB...