Beware - Emulex 10GbE Virtual Fabric Adapter II x8 PCI-E slot

Discussion in 'Networking' started by Stanza, Jun 22, 2014.

  1. Stanza

    Stanza Active Member

    Joined:
    Jan 11, 2014
    Messages:
    205
    Likes Received:
    40
    Just thought I would mention here

    I have a couple of IBM 49Y7942 eg rebranded Emulex OCE11102 cards

    Anyone wanting to use this card, please be aware it won't physically fit into a x8 PCI-E slot

    I thought I was going mad at 1st.....

    Tried it in a Dell Poweredge R310

    Fit in the right side (looking from front) slot......wouldn't fit in the left side.o_O

    Then I pulled it out and lined it up against the 4 port lan card I had in the left slot.... low and behold these are and extra 4pins longer than a x8 slot....

    So even tho they are advertised as a x8 card... you will have to place them into x16 slots

    Not sure if they will work in a normal x8 slot... with the back of the slot removed as some boards come these days??

    [​IMG]
     
    #1
    Last edited: Jul 29, 2018
    mervincm likes this.
  2. mervincm

    mervincm Active Member

    Joined:
    Jun 18, 2014
    Messages:
    141
    Likes Received:
    32
    can you provide any feedback on what SFP+ transceivers work for you in that card?
    these cards can be picked up for a fair price, but they never seem to come with the 850nm SFP+ transceivers.
     
    #2
  3. Stanza

    Stanza Active Member

    Joined:
    Jan 11, 2014
    Messages:
    205
    Likes Received:
    40
    #3
  4. mervincm

    mervincm Active Member

    Joined:
    Jun 18, 2014
    Messages:
    141
    Likes Received:
    32
    I was under the impression that the "locks" were applied on the socket side. Some attempts can be made on the transceiver side to fool these locks (specifically seen generic SFP that do work in locked Cisco gear) but I don't know of locks transceiver side.

    I need to pickup a dual 10G card that will work in a NAS, and my broadcom card does not..... so I am interested in your experience with the emulex here....
    I would appreciate your follow-up.
     
    #4
  5. Stanza

    Stanza Active Member

    Joined:
    Jan 11, 2014
    Messages:
    205
    Likes Received:
    40
    SFP's arrived

    Popped them in and they light up,

    Connected them with some old fibre cable I have around
    [​IMG]

    [​IMG]

    Just shows Link is Down on both ends

    Tried swapping 5 different cables, and also swapping one end Left to Right (eg so yellow is on left on each end) still link is down

    Transceivers light up, but I guess they are not supported? or the cable is the wrong type??
     
    #5
  6. mervincm

    mervincm Active Member

    Joined:
    Jun 18, 2014
    Messages:
    141
    Likes Received:
    32
    At that length those cables would be expected to work. I have used older less capable cables in short runs at 10G w/o issue.
    By light up, you mean that you have lights built into your transceiver (never seen this before) or are you talking about lights on the card/switch? My cards/switch only light up once a physical link has been established.

    I would offer to try one of them for you in my unlocked Mikrotik switch, but shipping to Canada might be more that the value of the SFP+

    Also I have not had to swap over any LC cables like that. You can hold a piece of paper up to a powered up cable, or a NIC, and see the tell-tale red light. lit side does to unlit side and you should be good.

    Also, are you going between two identical transceivers? there are multiple standards, designed for different distances etc, and they are not compatible to my understanding. The standard 10gig, short distance models are usually 850nm multimode.
     
    #6
  7. Stanza

    Stanza Active Member

    Joined:
    Jan 11, 2014
    Messages:
    205
    Likes Received:
    40
    By light up I mean, looking into the sfp... Can see the tx side is lit red

    Eg .... Dr evil voice..... Fricken laser beam..... ( led )

    No no link lights, the the actual LEDs do work....when doing the port identify thing in cards bios.

    So
    Bad sfp's
    Incompat sfp's
    ???

    Can you crossflash these ala sas HBA's to generic emulex? If so how and would that maybe remove vendor lockout on the sfp's

    As I notice when looking up the ibm part number for the card, they only list a few sfp's as supported.... And searching ebay...you find bucket loads of sfp with similar part numbers.... But not exactly.

    Eg
    Finistar.....xyz part number listed for these cards ends in last two letters -em

    Exact same part number for a dell ends in different two letters say -oc

    Probably really the same part... But card is looking for only a certain range in the "compat" part number... Anything else get locked out?

    Tho I would have thought if say a sfp wasn't supported the tx light wouldn't even fire up?

    .
     
    #7
  8. Mike

    Mike Member

    Joined:
    May 29, 2012
    Messages:
    482
    Likes Received:
    16
    Do not look into the sfp with your remaining eye. :rolleyes:
     
    #8
    Fritz likes this.
  9. s0lid

    s0lid Active Member

    Joined:
    Feb 25, 2013
    Messages:
    258
    Likes Received:
    33
    Short range (850nm) laser isn't that dangerous look at for second or two, but stay the hell away from the long range lasers (1200-1500nm).
     
    #9
  10. Stanza

    Stanza Active Member

    Joined:
    Jan 11, 2014
    Messages:
    205
    Likes Received:
    40
    #10
  11. Stanza

    Stanza Active Member

    Joined:
    Jan 11, 2014
    Messages:
    205
    Likes Received:
    40
    #11
  12. azev

    azev Active Member

    Joined:
    Jan 18, 2013
    Messages:
    673
    Likes Received:
    174
    So I had purchased one of these a while back and have similar problem. I tried flashing the firmware to the latest from emulex site with no luck.... To enable the link.. if you are running windows, you can download the driver and using emulex software bounce the port... I was able to get mine on that way. Regardless I was getting weird unidirectional traffic problem; I cant remember which way, but either tx or rx can barely push any data at all while another direction I can max 10gb link. Tried different fiber cable, through switches, twinax and result is pretty much the same.
     
    #12
  13. Stanza

    Stanza Active Member

    Joined:
    Jan 11, 2014
    Messages:
    205
    Likes Received:
    40
    Doh, guess I better install some windows and give that a whirl until my cables get here.
     
    #13
  14. Stanza

    Stanza Active Member

    Joined:
    Jan 11, 2014
    Messages:
    205
    Likes Received:
    40
    so how does one "bounce the port" ????

    .
     
    #14
  15. mervincm

    mervincm Active Member

    Joined:
    Jun 18, 2014
    Messages:
    141
    Likes Received:
    32
    I am seeing something like this as well. It's not consistent, and I thought it my switch. then I saw it pop up in a point to point connection. files copy 300-500 MB/sec in one direction and 60mbit/sec in the other.
     
    #15
  16. Stanza

    Stanza Active Member

    Joined:
    Jan 11, 2014
    Messages:
    205
    Likes Received:
    40
    What "mode" is the card running in? That's giving the problems?

    Eg my card seems to have 4 modes, the first 3 of which seem to only be for if the card is connected to a top of rack switch

    I am currently running it in pnic mode

    ==================

    Modes of operation

    The original Emulex 10GbE Virtual Fabric Adapter can be configured in pNIC or vNIC mode. The new Emulex 10GbE Virtual Fabric Adapter II now offers two vNIC modes of operation in addition to pNIC mode:
    • IBM Virtual Fabric Mode, which is the same as vNIC mode on original Emulex 10GbE Virtual Fabric Adapter. This mode only works in conjunction with IBM Virtual Fabric 10Gb Switch Module installed in I/O module bays 7 and 9 in the chassis. In this mode, the Emulex adapter communicated with the switch module to obtain vNIC parameters (using DCBX). Also, a special tag within each data packet is added and later removed by the NIC and switch for each vNIC group to maintain separation of the virtual channels.
      In vNIC mode, each physical port is divided into four virtual ports providing a total of eight virtual NICs per adapter. The default bandwidth for each vNIC is 2.5 Gbps. Bandwidth for each vNIC can be configured at the IBM switch from 100 Mbps to 10 Gbps, up to a total of 10 Gb per physical port. The vNICs can also be configured to have 0 bandwidth if you must allocate the available bandwidth to fewer than eight vNICs. In IBM Virtual Fabric Mode, you can change the bandwidth allocations through the IBM switch user interfaces without requiring a reboot of the server.

      When storage protocols are enabled on the Emulex 10GbE Virtual Fabric Adapter Advanced II, six ports are Ethernet, and two ports are either iSCSI or FCoE

    • Switch Independent Mode, where the adapter works with the following switches:
      • IBM Virtual Fabric 10Gb Switch Module
      • Cisco Nexus 4001I Switch Module
      • Brocade Converged 10GbE Switch Module
      • 10Gb Ethernet Pass-Thru Module and a top-of-rack (TOR) switch

      These switches must be installed in I/O module bays 7 and 9 in the chassis. Beyond the way that the adapter communicates with the switch, Switch Independent Mode offers the same capabilities as IBM Virtual Fabric Mode in terms of the number of vNICs and the bandwidth each can be configured to have. Switch Independent Mode extends the existing customer VLANs to the virtual NIC interfaces. The IEEE 802.1Q VLAN tag is essential to the separation of the vNIC groups by the NIC adapter or driver and the switch. The VLAN tags are added to the packet by the applications or drivers at each end station rather than by the switch.
    • In pNIC mode, the expansion card can operate as a standard 10 Gbps or 1 Gbps 2-port Ethernet expansion card. When in pNIC mode, the expansion card functions with any available 10 Gb switch or 10 Gb pass-thru module installed in I/O module bays 7 and 9.
      In pNIC mode, the Advanced adapter or the Advanced upgrade operates in traditional Converged Network Adapter (CNA) mode with two Ethernet ports and two storage ports (iSCSI or FCoE) available to the operating system..
     
    #16
  17. mervincm

    mervincm Active Member

    Joined:
    Jun 18, 2014
    Messages:
    141
    Likes Received:
    32
    I actually have broadcom cards
     
    #17
  18. azev

    azev Active Member

    Joined:
    Jan 18, 2013
    Messages:
    673
    Likes Received:
    174
    you can go to the nic management software and enable/disable the interface. The nic ports default on disabled and you have to enable it to get a link light.
     
    #18
  19. azev

    azev Active Member

    Joined:
    Jan 18, 2013
    Messages:
    673
    Likes Received:
    174
    the problem i am seeing is particular to the IBM NIC op posted above. I have a bunch of other emulex oneconnect adapter and have no problem pushing 10gb duplex using different tcp speedtest software.
     
    #19
  20. Stanza

    Stanza Active Member

    Joined:
    Jan 11, 2014
    Messages:
    205
    Likes Received:
    40
    no such luck here

    just link down all the time, guess I have to wait for the other cables to try again

    [​IMG]

    [​IMG]

    [​IMG]

    Edit, actually I just looked at the transceiver carton again.... On the bottom is labeled IBM p/n 46C3447

    Which according to this site

    IBM Redbooks | Emulex 10Gb Virtual Fabric Adapter for IBM System x (Withdrawn)

    Should be compatible??
     
    #20
    Last edited: Jun 29, 2014

Share This Page