Need help with starting 10gb setup

Discussion in 'Networking' started by TeleFragger, Dec 6, 2018.

  1. TeleFragger

    TeleFragger Member

    Oct 26, 2016
    Likes Received:
    any help greatly appreciated... warning... n00b at this ...

    Ok so I'm new to this and I have a test lab at work.

    I have 4 esxi 6.7 boxes - will get to them later
    a windows 10 pc (dell precision 5810)

    Our guys were getting rid of a mess of hardware so I grabbed it and figured time to play around.
    I stuck a mellanox card in the win10 box and drivers installed, albeit, MS drivers. but it seems to be good?
    I also connected a cable to the switch and the light on the switch lit up but in win10 the NIC says cable unattached.

    switch - Flextronics F-x430066 8 Port 4x SDR Infiniband
    cable - Mellanox Mcc4l30-300 Microgigacn Latch 0.3 M Infiniband Cable
    cards - HPE InfiniBand 4X DDR Conn-X PCI-E G2 Dual Port HCA (483514-B21)

    so few questions...
    what is needed on win10 side other than what I have done?

    if I put another card in a secondary pc, and just attach both to the switch, just give static IP's and that is it?

    so out of the manual... I get the green light with just the cable from pc to switch and nothing else to switch...

    The GREEN LED indicator to the left of each port will light when the physical connection is established (that is, when the unit is powered on and a cable is plugged into the port with a functioning port plugged into the other end of the connector)

    in Windows 10 dev manager - shows using Mellanox drivers signed by Microsoft
    ipoib6x.sys - 4/25/2018 ver. 5.50.14643.0

    C:\>mst status -v
    MST devices:
    mt26418_pci_cr0 bus:dev.fn=01:00.0
    mt26418_pciconf0 bus:dev.fn=01:00.0

    C:\>mlxfwmanager -d mt26418_pci_cr0 --query
    Querying Mellanox devices firmware ...
    FATAL - Can't find device id.
    FATAL - Can't find device id.
    Device #1:
    Device Type: N/A
    Part Number: --
    PCI Device Name: mt26418_pci_cr0
    Port1 MAC: N/A
    Port1 GUID: N/A
    Port2 MAC: N/A
    Port2 GUID: N/A
    Versions: Current Available
    FW --
    Status: Failed to open device
  2. TeleFragger

    TeleFragger Member

    Oct 26, 2016
    Likes Received:
    I learned a lot and have them working in win10 and esxi 6.7 as Ethernet... caveat is that I can only get the top port working and the bottom wont. even though ibstat states that there are 2 ports, windows sees 2 ports but plug cable in and no link lights.. ughhhh
    since I wasn't getting any replies here, I also posted elsewhere.. hope it isn't against the rules to post this and if so mod plz remove link...

    starting to play with 10gb - Mellanox and need help

    if you want to see all that ive done and where ive gone...
  3. Member

    Oct 6, 2017
    Likes Received:
    To get infiniband to work, you will need a fabric manager and an infinite amount of time to set the whole network up...
    Patrick likes this.
  4. Patrick

    Patrick Administrator
    Staff Member

    Dec 21, 2010
    Likes Received:
    I am going to give a bit of perspective here. You can get a 10GbE switch inexpensively. You can also hey SFP+ 10GbE cards inexpensively.

    Ethernet is significantly easier to setup and integrate than IB. At 56Gbps or higher IB has some advantages. Anything slower I would just do Ethernet.
    TeleFragger likes this.
  5. TeleFragger

    TeleFragger Member

    Oct 26, 2016
    Likes Received:
    so I am not doing IB at all.. I am doing IP and got it all going. Albeit not totally how I want as I want to use a switch but for now...

    you say
    can you elaborate with part numbers and prices so I can see what "inexpensive" is?

    so far I have $0 into it doing a pier to pier setup. My ESXI 6.7 box will only see these cards as ethernet, which is fine with me and it is working with my freenas box as well.

    I wouldn't mind upgrading to sfp+ but as stated... define inexpensive as everything I have seen (for 5 pc's) will run me about $500 at a minimum and my budget is maybe $150
Similar Threads: Need help
Forum Title Date
Networking vlan help needed Jan 17, 2019
Networking Dell Networking OS10 Help!!! Simple L2 help needed Nov 29, 2018
Networking Need some help installing pfsense or opnsense on HP T620 Plus [RESOLVED] Jul 7, 2018
Networking Need help determining if an Intel X550-T2 is real of fake. speed issues May 18, 2018
Networking Open Mesh A60 - Need Help with VLAN Mar 23, 2018

Share This Page