Beware - Emulex 10GbE Virtual Fabric Adapter II x8 PCI-E slot
- Thread starter Stanza
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Hmmm ok had another play tonight
thought I would try and find an older firmware, specifically from IBM for these cards.... not the Emulex firmware that they generally direct you to.
found a different firmware,
cards originally were
1st card was firmware version 4.2.412
2nd card was older at 3.x.xx
updated both originally with the emulex firmware ver 10.0.803.31
Both cards and transceivers wouldn't get a link
so now I found IBM Firmware 10.2.261.36-1
applied that in windows, connected port 0 to port 1 and got a link up
applied the same firmware to the 2nd card, installed ubuntu 12.04.........connect cable between the two servers (port 0 on each) and back to the link down status
fiddled around in the bios and changed the port status's to disabled, then back again to enabled........boom 10gbps link
exit bios and reboot....link light still on (good sign)
boot into ubuntu 12.04 live usb.......link is still up.
quick ping and a samba copy test to the windows box.......good it's working
so now installing ubuntu 12.04 properly to see what happens
pretty happy, even if it's only testing at present.... and lots to learn and configure yet. (kind of wish I found this before I forked out for the DAC cables that are on route....but meh I still need one more cable at minimum)
total cost so far
2 x IBM 49Y7942 = $100 + $8 postage
3 x SFP+ Transceivers = $31.63 + $6.15 postage
2 x LC>LC Cables Free already owned them
Total $145.78
3 x DAC Cables still to arrive $155.71 >>> more than the working lot above
Might have to offload a couple of the DAC cables to recoup some costs
thought I would try and find an older firmware, specifically from IBM for these cards.... not the Emulex firmware that they generally direct you to.
found a different firmware,
cards originally were
1st card was firmware version 4.2.412
2nd card was older at 3.x.xx
updated both originally with the emulex firmware ver 10.0.803.31
Both cards and transceivers wouldn't get a link
so now I found IBM Firmware 10.2.261.36-1
applied that in windows, connected port 0 to port 1 and got a link up
applied the same firmware to the 2nd card, installed ubuntu 12.04.........connect cable between the two servers (port 0 on each) and back to the link down status
fiddled around in the bios and changed the port status's to disabled, then back again to enabled........boom 10gbps link
exit bios and reboot....link light still on (good sign)
boot into ubuntu 12.04 live usb.......link is still up.
quick ping and a samba copy test to the windows box.......good it's working
so now installing ubuntu 12.04 properly to see what happens
pretty happy, even if it's only testing at present.... and lots to learn and configure yet. (kind of wish I found this before I forked out for the DAC cables that are on route....but meh I still need one more cable at minimum)
total cost so far
2 x IBM 49Y7942 = $100 + $8 postage
3 x SFP+ Transceivers = $31.63 + $6.15 postage
2 x LC>LC Cables Free already owned them
Total $145.78
3 x DAC Cables still to arrive $155.71
>>> more than the working lot aboveMight have to offload a couple of the DAC cables to recoup some costs
Quick test ubuntu 12.04 to windows 2012 essentials R2
iperf 2.0.5
Could only get it working one way....
windows as the server and ubuntu as the client....can't connect back the other way ???
Installed ubuntu 12.04 on both servers
iperf
update interval every 2 seconds
running for 20 seconds
packet size 1024k
running 6 concurrent threads
iperf -s -i 2 -t 20 -w 1024k -P 6
gives a pretty consistent
9.40 Gbits/s one direction
9.39 Gbits/sec the other
Server 1 to server 2
Code:
~$ iperf -c 192.168.1.83 -i 2 -t 20 -w 1024k -P 6
------------------------------------------------------------
Client connecting to 192.168.1.83, TCP port 5001
TCP window size: 256 KByte (WARNING: requested 1.00 MByte)
------------------------------------------------------------
[ 8] local 192.168.1.73 port 48023 connected with 192.168.1.83 port 5001
[ 3] local 192.168.1.73 port 48018 connected with 192.168.1.83 port 5001
[ 5] local 192.168.1.73 port 48020 connected with 192.168.1.83 port 5001
[ 4] local 192.168.1.73 port 48019 connected with 192.168.1.83 port 5001
[ 6] local 192.168.1.73 port 48021 connected with 192.168.1.83 port 5001
[ 7] local 192.168.1.73 port 48022 connected with 192.168.1.83 port 5001
[ ID] Interval Transfer Bandwidth
[ 8] 0.0- 2.0 sec 391 MBytes 1.64 Gbits/sec
[ 3] 0.0- 2.0 sec 376 MBytes 1.58 Gbits/sec
[ 5] 0.0- 2.0 sec 334 MBytes 1.40 Gbits/sec
[ 4] 0.0- 2.0 sec 385 MBytes 1.61 Gbits/sec
[ 6] 0.0- 2.0 sec 369 MBytes 1.55 Gbits/sec
[ 7] 0.0- 2.0 sec 383 MBytes 1.61 Gbits/sec
[SUM] 0.0- 2.0 sec 2.19 GBytes 9.39 Gbits/sec
[ 8] 2.0- 4.0 sec 376 MBytes 1.58 Gbits/sec
[ 3] 2.0- 4.0 sec 370 MBytes 1.55 Gbits/sec
[ 5] 2.0- 4.0 sec 376 MBytes 1.58 Gbits/sec
[ 4] 2.0- 4.0 sec 376 MBytes 1.58 Gbits/sec
[ 6] 2.0- 4.0 sec 366 MBytes 1.53 Gbits/sec
[ 7] 2.0- 4.0 sec 375 MBytes 1.57 Gbits/sec
[SUM] 2.0- 4.0 sec 2.19 GBytes 9.40 Gbits/sec
[ 8] 4.0- 6.0 sec 377 MBytes 1.58 Gbits/sec
[ 3] 4.0- 6.0 sec 362 MBytes 1.52 Gbits/sec
[ 5] 4.0- 6.0 sec 376 MBytes 1.57 Gbits/sec
[ 4] 4.0- 6.0 sec 375 MBytes 1.57 Gbits/sec
[ 6] 4.0- 6.0 sec 376 MBytes 1.58 Gbits/sec
[ 7] 4.0- 6.0 sec 376 MBytes 1.57 Gbits/sec
[SUM] 4.0- 6.0 sec 2.19 GBytes 9.40 Gbits/sec
[ 8] 6.0- 8.0 sec 375 MBytes 1.57 Gbits/sec
[ 3] 6.0- 8.0 sec 374 MBytes 1.57 Gbits/sec
[ 5] 6.0- 8.0 sec 376 MBytes 1.58 Gbits/sec
[ 4] 6.0- 8.0 sec 374 MBytes 1.57 Gbits/sec
[ 6] 6.0- 8.0 sec 368 MBytes 1.55 Gbits/sec
[ 7] 6.0- 8.0 sec 373 MBytes 1.56 Gbits/sec
[SUM] 6.0- 8.0 sec 2.19 GBytes 9.40 Gbits/sec
[ 8] 8.0-10.0 sec 371 MBytes 1.56 Gbits/sec
[ 3] 8.0-10.0 sec 375 MBytes 1.57 Gbits/sec
[ 5] 8.0-10.0 sec 375 MBytes 1.57 Gbits/sec
[ 4] 8.0-10.0 sec 372 MBytes 1.56 Gbits/sec
[ 6] 8.0-10.0 sec 372 MBytes 1.56 Gbits/sec
[ 7] 8.0-10.0 sec 375 MBytes 1.57 Gbits/sec
[SUM] 8.0-10.0 sec 2.19 GBytes 9.40 Gbits/sec
[ 8] 10.0-12.0 sec 362 MBytes 1.52 Gbits/sec
[ 3] 10.0-12.0 sec 376 MBytes 1.58 Gbits/sec
[ 5] 10.0-12.0 sec 377 MBytes 1.58 Gbits/sec
[ 4] 10.0-12.0 sec 375 MBytes 1.57 Gbits/sec
[ 6] 10.0-12.0 sec 375 MBytes 1.57 Gbits/sec
[ 7] 10.0-12.0 sec 376 MBytes 1.58 Gbits/sec
[SUM] 10.0-12.0 sec 2.19 GBytes 9.40 Gbits/sec
[ 8] 12.0-14.0 sec 376 MBytes 1.58 Gbits/sec
[ 3] 12.0-14.0 sec 378 MBytes 1.58 Gbits/sec
[ 5] 12.0-14.0 sec 375 MBytes 1.57 Gbits/sec
[ 4] 12.0-14.0 sec 370 MBytes 1.55 Gbits/sec
[ 6] 12.0-14.0 sec 369 MBytes 1.55 Gbits/sec
[ 7] 12.0-14.0 sec 372 MBytes 1.56 Gbits/sec
[SUM] 12.0-14.0 sec 2.19 GBytes 9.40 Gbits/sec
[ 8] 14.0-16.0 sec 374 MBytes 1.57 Gbits/sec
[ 3] 14.0-16.0 sec 369 MBytes 1.55 Gbits/sec
[ 5] 14.0-16.0 sec 376 MBytes 1.58 Gbits/sec
[ 4] 14.0-16.0 sec 376 MBytes 1.58 Gbits/sec
[ 6] 14.0-16.0 sec 372 MBytes 1.56 Gbits/sec
[ 7] 14.0-16.0 sec 375 MBytes 1.57 Gbits/sec
[SUM] 14.0-16.0 sec 2.19 GBytes 9.40 Gbits/sec
[ 8] 16.0-18.0 sec 372 MBytes 1.56 Gbits/sec
[ 3] 16.0-18.0 sec 371 MBytes 1.56 Gbits/sec
[ 5] 16.0-18.0 sec 374 MBytes 1.57 Gbits/sec
[ 4] 16.0-18.0 sec 374 MBytes 1.57 Gbits/sec
[ 6] 16.0-18.0 sec 376 MBytes 1.58 Gbits/sec
[ 7] 16.0-18.0 sec 374 MBytes 1.57 Gbits/sec
[SUM] 16.0-18.0 sec 2.19 GBytes 9.40 Gbits/sec
[ 8] 18.0-20.0 sec 381 MBytes 1.60 Gbits/sec
[ 8] 0.0-20.0 sec 3.67 GBytes 1.58 Gbits/sec
[ 3] 0.0-20.0 sec 3.64 GBytes 1.56 Gbits/sec
[ 5] 0.0-20.0 sec 3.63 GBytes 1.56 Gbits/sec
[ 4] 0.0-20.0 sec 3.66 GBytes 1.57 Gbits/sec
[ 6] 0.0-20.0 sec 3.63 GBytes 1.56 Gbits/sec
[ 7] 18.0-20.0 sec 362 MBytes 1.52 Gbits/sec
[ 7] 0.0-20.0 sec 3.65 GBytes 1.57 Gbits/sec
[SUM] 0.0-20.0 sec 21.9 GBytes 9.40 Gbits/sec
Code:
~$ iperf -c 192.168.1.73 -i 2 -t 20 -w 1024k -P 6
------------------------------------------------------------
Client connecting to 192.168.1.73, TCP port 5001
TCP window size: 256 KByte (WARNING: requested 1.00 MByte)
------------------------------------------------------------
[ 8] local 192.168.1.83 port 53557 connected with 192.168.1.73 port 5001
[ 4] local 192.168.1.83 port 53552 connected with 192.168.1.73 port 5001
[ 3] local 192.168.1.83 port 53553 connected with 192.168.1.73 port 5001
[ 5] local 192.168.1.83 port 53554 connected with 192.168.1.73 port 5001
[ 6] local 192.168.1.83 port 53555 connected with 192.168.1.73 port 5001
[ 7] local 192.168.1.83 port 53556 connected with 192.168.1.73 port 5001
[ ID] Interval Transfer Bandwidth
[ 8] 0.0- 2.0 sec 441 MBytes 1.85 Gbits/sec
[ 4] 0.0- 2.0 sec 431 MBytes 1.81 Gbits/sec
[ 3] 0.0- 2.0 sec 438 MBytes 1.84 Gbits/sec
[ 5] 0.0- 2.0 sec 457 MBytes 1.92 Gbits/sec
[ 6] 0.0- 2.0 sec 205 MBytes 860 Mbits/sec
[ 7] 0.0- 2.0 sec 266 MBytes 1.11 Gbits/sec
[SUM] 0.0- 2.0 sec 2.19 GBytes 9.39 Gbits/sec
[ 8] 2.0- 4.0 sec 442 MBytes 1.85 Gbits/sec
[ 4] 2.0- 4.0 sec 429 MBytes 1.80 Gbits/sec
[ 3] 2.0- 4.0 sec 441 MBytes 1.85 Gbits/sec
[ 5] 2.0- 4.0 sec 444 MBytes 1.86 Gbits/sec
[ 6] 2.0- 4.0 sec 236 MBytes 989 Mbits/sec
[ 7] 2.0- 4.0 sec 247 MBytes 1.04 Gbits/sec
[SUM] 2.0- 4.0 sec 2.19 GBytes 9.39 Gbits/sec
[ 8] 4.0- 6.0 sec 441 MBytes 1.85 Gbits/sec
[ 4] 4.0- 6.0 sec 447 MBytes 1.88 Gbits/sec
[ 3] 4.0- 6.0 sec 438 MBytes 1.84 Gbits/sec
[ 5] 4.0- 6.0 sec 440 MBytes 1.85 Gbits/sec
[ 6] 4.0- 6.0 sec 236 MBytes 988 Mbits/sec
[ 7] 4.0- 6.0 sec 237 MBytes 994 Mbits/sec
[SUM] 4.0- 6.0 sec 2.19 GBytes 9.39 Gbits/sec
[ 8] 6.0- 8.0 sec 440 MBytes 1.84 Gbits/sec
[ 4] 6.0- 8.0 sec 437 MBytes 1.83 Gbits/sec
[ 3] 6.0- 8.0 sec 441 MBytes 1.85 Gbits/sec
[ 5] 6.0- 8.0 sec 447 MBytes 1.87 Gbits/sec
[ 6] 6.0- 8.0 sec 236 MBytes 988 Mbits/sec
[ 7] 6.0- 8.0 sec 239 MBytes 1.00 Gbits/sec
[SUM] 6.0- 8.0 sec 2.19 GBytes 9.39 Gbits/sec
[ 8] 8.0-10.0 sec 454 MBytes 1.90 Gbits/sec
[ 4] 8.0-10.0 sec 432 MBytes 1.81 Gbits/sec
[ 3] 8.0-10.0 sec 430 MBytes 1.80 Gbits/sec
[ 5] 8.0-10.0 sec 448 MBytes 1.88 Gbits/sec
[ 6] 8.0-10.0 sec 245 MBytes 1.03 Gbits/sec
[ 7] 8.0-10.0 sec 230 MBytes 967 Mbits/sec
[SUM] 8.0-10.0 sec 2.19 GBytes 9.39 Gbits/sec
[ 8] 10.0-12.0 sec 448 MBytes 1.88 Gbits/sec
[ 4] 10.0-12.0 sec 437 MBytes 1.83 Gbits/sec
[ 3] 10.0-12.0 sec 444 MBytes 1.86 Gbits/sec
[ 5] 10.0-12.0 sec 449 MBytes 1.88 Gbits/sec
[ 6] 10.0-12.0 sec 232 MBytes 975 Mbits/sec
[ 7] 10.0-12.0 sec 229 MBytes 961 Mbits/sec
[SUM] 10.0-12.0 sec 2.19 GBytes 9.39 Gbits/sec
[ 8] 12.0-14.0 sec 432 MBytes 1.81 Gbits/sec
[ 4] 12.0-14.0 sec 433 MBytes 1.82 Gbits/sec
[ 3] 12.0-14.0 sec 445 MBytes 1.86 Gbits/sec
[ 5] 12.0-14.0 sec 446 MBytes 1.87 Gbits/sec
[ 6] 12.0-14.0 sec 240 MBytes 1.01 Gbits/sec
[ 7] 12.0-14.0 sec 243 MBytes 1.02 Gbits/sec
[SUM] 12.0-14.0 sec 2.19 GBytes 9.39 Gbits/sec
[ 8] 14.0-16.0 sec 443 MBytes 1.86 Gbits/sec
[ 4] 14.0-16.0 sec 437 MBytes 1.83 Gbits/sec
[ 3] 14.0-16.0 sec 441 MBytes 1.85 Gbits/sec
[ 5] 14.0-16.0 sec 444 MBytes 1.86 Gbits/sec
[ 6] 14.0-16.0 sec 234 MBytes 984 Mbits/sec
[ 7] 14.0-16.0 sec 240 MBytes 1.00 Gbits/sec
[SUM] 14.0-16.0 sec 2.19 GBytes 9.39 Gbits/sec
[ 8] 16.0-18.0 sec 441 MBytes 1.85 Gbits/sec
[ 4] 16.0-18.0 sec 429 MBytes 1.80 Gbits/sec
[ 3] 16.0-18.0 sec 444 MBytes 1.86 Gbits/sec
[ 5] 16.0-18.0 sec 443 MBytes 1.86 Gbits/sec
[ 6] 16.0-18.0 sec 242 MBytes 1.02 Gbits/sec
[ 7] 16.0-18.0 sec 241 MBytes 1.01 Gbits/sec
[SUM] 16.0-18.0 sec 2.19 GBytes 9.39 Gbits/sec
[ 8] 18.0-20.0 sec 450 MBytes 1.89 Gbits/sec
[ 8] 0.0-20.0 sec 4.33 GBytes 1.86 Gbits/sec
[ 4] 0.0-20.0 sec 4.24 GBytes 1.82 Gbits/sec
[ 3] 0.0-20.0 sec 4.30 GBytes 1.84 Gbits/sec
[ 5] 0.0-20.0 sec 4.36 GBytes 1.87 Gbits/sec
[ 6] 18.0-20.0 sec 245 MBytes 1.03 Gbits/sec
[ 6] 0.0-20.0 sec 2.30 GBytes 986 Mbits/sec
[ 7] 18.0-20.0 sec 236 MBytes 990 Mbits/sec
[ 7] 0.0-20.0 sec 2.35 GBytes 1.01 Gbits/sec
[SUM] 0.0-20.0 sec 21.9 GBytes 9.39 Gbits/secShould mention,
Both servers are IBM X3400 M3, and the cards are in the V2 x16 slots (as they don't fit anywhere else.)
Maybe IBM actually uses the extra pins on these for something? hence the reason you had fast one direction / slow the other problems?
Tried iperf again with the MTU bumped up to 9000
Speeds up a little to 9.90 Gbits/sec
which I believe is probably max ??
Both servers are IBM X3400 M3, and the cards are in the V2 x16 slots (as they don't fit anywhere else.)
Maybe IBM actually uses the extra pins on these for something? hence the reason you had fast one direction / slow the other problems?
Tried iperf again with the MTU bumped up to 9000
Speeds up a little to 9.90 Gbits/sec
which I believe is probably max ??
Last edited:
When I get a chance, I will pop one in a dell R310 and also a supermicro to try and figure out if the pin issue is the problem.
Although I am still having to disable / enable the port every time they are fired up ..... Must also work that issue out....
.
Although I am still having to disable / enable the port every time they are fired up ..... Must also work that issue out....
.
OK my DAC cables finally arrived
also received a REAL Emulex card (not IBM one with extra pins) to try out.... (this one came with 2 Transceivers)
Still having troubles getting a link happening consistently
Connect DAC cable to port 1 of card one in server 1
Connect DAC cable to port 1 of card one in server 2
No link
Go into bios, disable both ports and re enable them (on both servers) and get a link
pull out DAC cable, swap for old Transceivers........ insert fibre.......get a link
Pull out old transceivers.....pop in new ones that came with above card......get a link
power off one server, power it back on.......link stays down
have to go back into bios and do the disable / enable dance again....to get them to link back up. Grrr
Can't be doing that every time I power down a server.......which being a home lab setup will be quite often
how does one get these suckers to keep in a link up mode?
Bios has some power saving settings....... guess I will try next to max out everything and see if thats the problem.... eg cards are being forced into a low power / off state etc......and the disable / enable port trick actually wakes the cards back up??
Also trying to figure out how to install Emulex HBAnywhere management utils on Ubuntu...... as reading the docs, you can control the ports in the cli that way? tho it all seems to be Redhat based RPM packages.
Maybe thats one way around the problem?.... a script at startup to disable / enable the ports to get a link going.
anyone else having a similar issue? found a way around the problem?........or has installed the software ?
also received a REAL Emulex card (not IBM one with extra pins) to try out.... (this one came with 2 Transceivers)
Still having troubles getting a link happening consistently
Connect DAC cable to port 1 of card one in server 1
Connect DAC cable to port 1 of card one in server 2
No link
Go into bios, disable both ports and re enable them (on both servers) and get a link
pull out DAC cable, swap for old Transceivers........ insert fibre.......get a link
Pull out old transceivers.....pop in new ones that came with above card......get a link
power off one server, power it back on.......link stays down
have to go back into bios and do the disable / enable dance again....to get them to link back up. Grrr
Can't be doing that every time I power down a server.......which being a home lab setup will be quite often
how does one get these suckers to keep in a link up mode?
Bios has some power saving settings....... guess I will try next to max out everything and see if thats the problem.... eg cards are being forced into a low power / off state etc......and the disable / enable port trick actually wakes the cards back up??
Also trying to figure out how to install Emulex HBAnywhere management utils on Ubuntu...... as reading the docs, you can control the ports in the cli that way? tho it all seems to be Redhat based RPM packages.
Maybe thats one way around the problem?.... a script at startup to disable / enable the ports to get a link going.
anyone else having a similar issue? found a way around the problem?........or has installed the software ?
Just an info for everybody facing the same problem (no link on Emulex 10GbE Custom Adapter for IBM System X (SBB 49Y7940)).
Under vmware ESXi use following command in shell:
esxcli network nic set -S 10000 -D full -n vmnic5
it brings interface (link) up.
Replace vmnic5 with your nic ID, shown by the command
esxcli network nic list
(i'm running ESXi 5.1)
Under Windows Server 2012 it's enough to disable/enable port in OneConnect Manager.
Under vmware ESXi use following command in shell:
esxcli network nic set -S 10000 -D full -n vmnic5
it brings interface (link) up.
Replace vmnic5 with your nic ID, shown by the command
esxcli network nic list
(i'm running ESXi 5.1)
Under Windows Server 2012 it's enough to disable/enable port in OneConnect Manager.
This thread saved me a lot of time and effort so I thought I would give something back. I also experienced the same issues with the link not coming up in Windows 2012 R2. I have Two 49Y4202 directly connected via a 5 meter Twinax DAC passive cable.
Here is a small and very, very basic batch file you can run at startup to enable the link . It's not optimal but it works.
Prerequisites
Install the OneCommand Manager GUI on both windows 2012 servers (Link as of 9/29/2016). You can also install the CLI version. I tested this on the GUI version. It should install to c:\Program Files\Emulex. The executable we need is located in c:\Program Files\Emulex\Util\OCManager>HbaCmd.exe. I would also highly recommend reading the Windows CLI PDF located here. It has details on all of the commands you can run.
Get the Mac address of each port by entering the following into an elevated command prompt. Change x.x.x.x to the ip address of the Onecommand server. If you run this command locally it will tell you to remove the h=x.x.x.x. Obviously you need to do this on both sides of the link in order to collect all 4 Mac addresses. Assuming you have two 2 port NICS.
c:\Program Files\Emulex\Util\OCManager>HbaCmd.exe h=x.x.x.x listhba
You should see something like this:
Manageable HBA List
Permanent MAC : 00-00-00-00-00-01
Current MAC : 00-00-00-00-00-01
Flags : 00000710
Host Name : xxxxxxxx
Mfg : Emulex Corporation
Serial No. : xxxxxxxxxxxxxxxxxx
Port Number : 0
Mode : Initiator
PCI Bus Number : 2
PCI Function : 0
Port Type : NIC
Model : 49Y7941
Discovery : RM
Permanent MAC : 00-00-00-00-00-02
Current MAC : 00-00-00-00-00-02
Flags : 00000710
Host Name : xxxxxxxx
Mfg : Emulex Corporation
Serial No. : xxxxxxxxxxxxxxxxxxxx
Port Number : 1
Mode : Initiator
PCI Bus Number : 2
PCI Function : 1
Port Type : NIC
Model : 49Y7941
Discovery : RM
Create a batch file such as enable_10GB.bat or whatever you want to call it.
Now that we have the MAC address of each port on each side of the link we can build the script.
Paste the following code into the script and save it as a .bat file You can leave out the rem statements if you desire.
@ECHO OFF
cd c:\scripts\emulex\
Rem "Make sure you have installed onecommand on both windows 2012 servers"
rem "To get the mac adress of each port run c:\Program Files\Emulex\Util\OCManager>HbaCmd.exe h=x.x.x.x listhba. Replace x.x.x.x with the ip of the remote onecommand server. Note the Permanent MAC of each controller port"
rem "Replace the 00-00-00-00-00-00 with the mac of the port1, port2, port3, port4. The 0 at the end disables the port and the 1 enables the port"
"C:\Program Files\Emulex\Util\OCManager\hbacmd" SetPortEnabled 00-00-00-00-00-01 0
"C:\Program Files\Emulex\Util\OCManager\hbacmd" SetPortEnabled 00-00-00-00-00-01 1
"C:\Program Files\Emulex\Util\OCManager\hbacmd" SetPortEnabled 00-00-00-00-00-02 0
"C:\Program Files\Emulex\Util\OCManager\hbacmd" SetPortEnabled 00-00-00-00-00-02 1
"C:\Program Files\Emulex\Util\OCManager\hbacmd" h=x.x.x.x SetPortEnabled 00-00-00-00-00-03 0
"C:\Program Files\Emulex\Util\OCManager\hbacmd" h=x.x.x.x SetPortEnabled 00-00-00-00-00-03 1
"C:\Program Files\Emulex\Util\OCManager\hbacmd" h=x.x.x.x SetPortEnabled 00-00-00-00-00-04 0
"C:\Program Files\Emulex\Util\OCManager\hbacmd" h=x.x.x.x SetPortEnabled 00-00-00-00-00-04 1
Copy the batch file to wherever you like. IE c:\scripts\emulex\.
Create a Basic Task scheduler task
Choose When computer starts Task Trigger
Choose Start a Program for the Action
Browse to wherever you stored the script. c:\scripts\emulex\ in this example. Leave arguments blank
Click the box to Open Properties dialog for this task when I click finish
Click Finish
Enter the username\password of the user to run the batch file. The user has to have admin rights.
In the advanced Settings area click the Delay task for up to (Random delay) and change the setting to 1 minute. This will give the onecommand server time to start.
Hit ok.
Power off the server and then turn it back on. The link stays up for me if I simply reboot either of the servers. It doesnt' come up only after a power off of either server. After one minute the link should come up on it's own.
Do the exact same steps to the 2nd server. Be sure to swap the IP's and mac addresses.
The other cool thing you can do is install the onemanager on your desktop. Change the script above to include the h=x.x.x.x for all of your servers and then run the script from your desktop. This will disable and enable all 4 ports. This is how I initially tested the script. Read the manual, there are some cool security features you can enable also. For obvious reasons, you probably don't want anyone on your network to be able to run these commands. In my home lab it's not a problem.
Here is a small and very, very basic batch file you can run at startup to enable the link . It's not optimal but it works.
Prerequisites
Install the OneCommand Manager GUI on both windows 2012 servers (Link as of 9/29/2016). You can also install the CLI version. I tested this on the GUI version. It should install to c:\Program Files\Emulex. The executable we need is located in c:\Program Files\Emulex\Util\OCManager>HbaCmd.exe. I would also highly recommend reading the Windows CLI PDF located here. It has details on all of the commands you can run.
Get the Mac address of each port by entering the following into an elevated command prompt. Change x.x.x.x to the ip address of the Onecommand server. If you run this command locally it will tell you to remove the h=x.x.x.x. Obviously you need to do this on both sides of the link in order to collect all 4 Mac addresses. Assuming you have two 2 port NICS.
c:\Program Files\Emulex\Util\OCManager>HbaCmd.exe h=x.x.x.x listhba
You should see something like this:
Manageable HBA List
Permanent MAC : 00-00-00-00-00-01
Current MAC : 00-00-00-00-00-01
Flags : 00000710
Host Name : xxxxxxxx
Mfg : Emulex Corporation
Serial No. : xxxxxxxxxxxxxxxxxx
Port Number : 0
Mode : Initiator
PCI Bus Number : 2
PCI Function : 0
Port Type : NIC
Model : 49Y7941
Discovery : RM
Permanent MAC : 00-00-00-00-00-02
Current MAC : 00-00-00-00-00-02
Flags : 00000710
Host Name : xxxxxxxx
Mfg : Emulex Corporation
Serial No. : xxxxxxxxxxxxxxxxxxxx
Port Number : 1
Mode : Initiator
PCI Bus Number : 2
PCI Function : 1
Port Type : NIC
Model : 49Y7941
Discovery : RM
Create a batch file such as enable_10GB.bat or whatever you want to call it.
Now that we have the MAC address of each port on each side of the link we can build the script.
Paste the following code into the script and save it as a .bat file You can leave out the rem statements if you desire.
@ECHO OFF
cd c:\scripts\emulex\
Rem "Make sure you have installed onecommand on both windows 2012 servers"
rem "To get the mac adress of each port run c:\Program Files\Emulex\Util\OCManager>HbaCmd.exe h=x.x.x.x listhba. Replace x.x.x.x with the ip of the remote onecommand server. Note the Permanent MAC of each controller port"
rem "Replace the 00-00-00-00-00-00 with the mac of the port1, port2, port3, port4. The 0 at the end disables the port and the 1 enables the port"
"C:\Program Files\Emulex\Util\OCManager\hbacmd" SetPortEnabled 00-00-00-00-00-01 0
"C:\Program Files\Emulex\Util\OCManager\hbacmd" SetPortEnabled 00-00-00-00-00-01 1
"C:\Program Files\Emulex\Util\OCManager\hbacmd" SetPortEnabled 00-00-00-00-00-02 0
"C:\Program Files\Emulex\Util\OCManager\hbacmd" SetPortEnabled 00-00-00-00-00-02 1
"C:\Program Files\Emulex\Util\OCManager\hbacmd" h=x.x.x.x SetPortEnabled 00-00-00-00-00-03 0
"C:\Program Files\Emulex\Util\OCManager\hbacmd" h=x.x.x.x SetPortEnabled 00-00-00-00-00-03 1
"C:\Program Files\Emulex\Util\OCManager\hbacmd" h=x.x.x.x SetPortEnabled 00-00-00-00-00-04 0
"C:\Program Files\Emulex\Util\OCManager\hbacmd" h=x.x.x.x SetPortEnabled 00-00-00-00-00-04 1
Copy the batch file to wherever you like. IE c:\scripts\emulex\.
Create a Basic Task scheduler task
Choose When computer starts Task Trigger
Choose Start a Program for the Action
Browse to wherever you stored the script. c:\scripts\emulex\ in this example. Leave arguments blank
Click the box to Open Properties dialog for this task when I click finish
Click Finish
Enter the username\password of the user to run the batch file. The user has to have admin rights.
In the advanced Settings area click the Delay task for up to (Random delay) and change the setting to 1 minute. This will give the onecommand server time to start.
Hit ok.
Power off the server and then turn it back on. The link stays up for me if I simply reboot either of the servers. It doesnt' come up only after a power off of either server. After one minute the link should come up on it's own.
Do the exact same steps to the 2nd server. Be sure to swap the IP's and mac addresses.
The other cool thing you can do is install the onemanager on your desktop. Change the script above to include the h=x.x.x.x for all of your servers and then run the script from your desktop. This will disable and enable all 4 ports. This is how I initially tested the script. Read the manual, there are some cool security features you can enable also. For obvious reasons, you probably don't want anyone on your network to be able to run these commands. In my home lab it's not a problem.
Last edited:
@Stanza I know this is a really old thread.... I just purchased one of these Emulex 10G cards, and am having a problem getting it running. Do you happen to have a link to the firmware you updated the card to? Or even if you have the binary package, that would be great!
Thanks!
I found this page on howto get these bastards to link up in esxi 5.5
Fixing Emulex 10GbE on ESXi 5.5
Seems they dont like the elxnet driver... and you have to make it run with the standard be2net driver...
removing the elxnet driver works, and the link comes up now at boot in esxi 5.5
command at the cli worked for me
Code:
esxcli software vib remove --vibname elxnetDear Stanza,
I have been trying to get the same card working but failed. The ethtool reported that there is no link detected. You have mentioned that you did something in the bios to get the link up. Much appreciated if you could share with us please.
I am using Ubuntu 16.04. The driver is be2net. The card is from IBM and it is a OC10 series with fw 10.0.803.31. I am using a Quanta LB6M switch and CISCO SFP-H10GB-CU3M 3M TWINAX COPPER cable.
Thank you in advance.
Regards,
Terry
I have been trying to get the same card working but failed. The ethtool reported that there is no link detected. You have mentioned that you did something in the bios to get the link up. Much appreciated if you could share with us please.
I am using Ubuntu 16.04. The driver is be2net. The card is from IBM and it is a OC10 series with fw 10.0.803.31. I am using a Quanta LB6M switch and CISCO SFP-H10GB-CU3M 3M TWINAX COPPER cable.
Thank you in advance.
Regards,
Terry