Need help with starting 10gb setup

Discussion in 'Networking' started by TeleFragger, Dec 6, 2018.

  1. TeleFragger

    TeleFragger Member

    Oct 26, 2016
    Likes Received:
    any help greatly appreciated... warning... n00b at this ...

    Ok so I'm new to this and I have a test lab at work.

    I have 4 esxi 6.7 boxes - will get to them later
    a windows 10 pc (dell precision 5810)

    Our guys were getting rid of a mess of hardware so I grabbed it and figured time to play around.
    I stuck a mellanox card in the win10 box and drivers installed, albeit, MS drivers. but it seems to be good?
    I also connected a cable to the switch and the light on the switch lit up but in win10 the NIC says cable unattached.

    switch - Flextronics F-x430066 8 Port 4x SDR Infiniband
    cable - Mellanox Mcc4l30-300 Microgigacn Latch 0.3 M Infiniband Cable
    cards - HPE InfiniBand 4X DDR Conn-X PCI-E G2 Dual Port HCA (483514-B21)

    so few questions...
    what is needed on win10 side other than what I have done?

    if I put another card in a secondary pc, and just attach both to the switch, just give static IP's and that is it?

    so out of the manual... I get the green light with just the cable from pc to switch and nothing else to switch...

    The GREEN LED indicator to the left of each port will light when the physical connection is established (that is, when the unit is powered on and a cable is plugged into the port with a functioning port plugged into the other end of the connector)

    in Windows 10 dev manager - shows using Mellanox drivers signed by Microsoft
    ipoib6x.sys - 4/25/2018 ver. 5.50.14643.0

    C:\>mst status -v
    MST devices:
    mt26418_pci_cr0 bus:dev.fn=01:00.0
    mt26418_pciconf0 bus:dev.fn=01:00.0

    C:\>mlxfwmanager -d mt26418_pci_cr0 --query
    Querying Mellanox devices firmware ...
    FATAL - Can't find device id.
    FATAL - Can't find device id.
    Device #1:
    Device Type: N/A
    Part Number: --
    PCI Device Name: mt26418_pci_cr0
    Port1 MAC: N/A
    Port1 GUID: N/A
    Port2 MAC: N/A
    Port2 GUID: N/A
    Versions: Current Available
    FW --
    Status: Failed to open device
  2. TeleFragger

    TeleFragger Member

    Oct 26, 2016
    Likes Received:
    I learned a lot and have them working in win10 and esxi 6.7 as Ethernet... caveat is that I can only get the top port working and the bottom wont. even though ibstat states that there are 2 ports, windows sees 2 ports but plug cable in and no link lights.. ughhhh
    since I wasn't getting any replies here, I also posted elsewhere.. hope it isn't against the rules to post this and if so mod plz remove link...

    starting to play with 10gb - Mellanox and need help

    if you want to see all that ive done and where ive gone...
  3. Member

    Oct 6, 2017
    Likes Received:
    To get infiniband to work, you will need a fabric manager and an infinite amount of time to set the whole network up...
    Patrick likes this.
  4. Patrick

    Patrick Administrator
    Staff Member

    Dec 21, 2010
    Likes Received:
    I am going to give a bit of perspective here. You can get a 10GbE switch inexpensively. You can also hey SFP+ 10GbE cards inexpensively.

    Ethernet is significantly easier to setup and integrate than IB. At 56Gbps or higher IB has some advantages. Anything slower I would just do Ethernet.
    TeleFragger likes this.
  5. TeleFragger

    TeleFragger Member

    Oct 26, 2016
    Likes Received:
    so I am not doing IB at all.. I am doing IP and got it all going. Albeit not totally how I want as I want to use a switch but for now...

    you say
    can you elaborate with part numbers and prices so I can see what "inexpensive" is?

    so far I have $0 into it doing a pier to pier setup. My ESXI 6.7 box will only see these cards as ethernet, which is fine with me and it is working with my freenas box as well.

    I wouldn't mind upgrading to sfp+ but as stated... define inexpensive as everything I have seen (for 5 pc's) will run me about $500 at a minimum and my budget is maybe $150
Similar Threads: Need help
Forum Title Date
Networking Need help on Voltaire VLT-4036E - 30034 May 10, 2019
Networking Need help with making reflashed HP / MCX354A work at 40Gb/s Apr 10, 2019
Networking Need help with HP Virtual Connect and iSCSI Mar 30, 2019
Networking Upgrading to 40gbE need some help Mar 1, 2019
Networking vlan help needed Jan 17, 2019

Share This Page