interesting quanta switch 48x gbe 4x sfp+

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
N

namike

Member
Sep 2, 2014
70
18
8
43
This is an odd switch port configuration, to say the least. I would simplify the configuration of your switch config as much as possible, IE remove pvid, vlan participation and tagging. Move everything back to default/VLAN 1. Leave your vSwitches as default as well.
 
E

Emulsifide

Active Member
Dec 1, 2014
212
93
28
namike said:
This is an odd switch port configuration, to say the least. I would simplify the configuration of your switch config as much as possible, IE remove pvid, vlan participation and tagging. Move everything back to default/VLAN 1. Leave your vSwitches as default as well.
Click to expand...
Yep, I've tried that. As h0tw1r3 has already shown, the switch came stock with everything flat on VLAN 1 with the following config:

Code: 
interface 0/49
no auto-negotiate
exit

interface 0/51
no auto-negotiate
exit
The vSwitches were left as the default as well (untagged). I literally tried every combination I could think of last night with the same results. The fact that no MACs are coming through the interfaces, I'm wondering if the switch's transceivers are not playing well with the Brocade ones. I just had two more Brocade transceivers show up in the mail. I'll try plugging those in and seeing if I get different results.
 
N

namike

Member
Sep 2, 2014
70
18
8
43
Emulsifide said:
Yep, I've tried that. As h0tw1r3 has already shown, the switch came stock with everything flat on VLAN 1 with the following config:

Code: 
interface 0/49
no auto-negotiate
exit

interface 0/51
no auto-negotiate
exit
The vSwitches were left as the default as well (untagged). I literally tried every combination I could think of last night with the same results. The fact that no MACs are coming through the interfaces, I'm wondering if the switch's transceivers are not playing well with the Brocade ones. I just had two more Brocade transceivers show up in the mail. I'll try plugging those in and seeing if I get different results.
Click to expand...
Yeah, if you are not getting any MAC addresses from your servers listed in the switch MAC table something is still weird. I see you just got some new transceivers, can you also try the 1g copper ports just to rule out the fiber cabling/transceivers?
 
E

Emulsifide

Active Member
Dec 1, 2014
212
93
28
namike said:
Yeah, if you are not getting any MAC addresses from your servers listed in the switch MAC table something is still weird. I see you just got some new transceivers, can you also try the 1g copper ports just to rule out the fiber cabling/transceivers?
Click to expand...
Thanks for the quick reply and the fact that you're willing to help. The copper ports work fine since I'm telnetted into the switch right now from port 0/1 being connected to the rest of my infrastructure. I just tried the Brocade transceivers I just got in the mail with the same results. Here's the entire workflow I did and diagnosis:

Code: 
(FASTPATH Routing) #show port all

               Admin   Physical   Physical   Link   Link    LACP   Actor
Intf   Type    Mode    Mode       Status   Status  Trap    Mode   Timeout
------ ------ ------- ---------- ---------- ------ ------- ------ --------
0/1           Enable  Auto       1000 Full  Up     Enable  Enable long
0/2           Enable  Auto                  Down   Enable  Enable long
0/3           Enable  Auto                  Down   Enable  Enable long
0/4           Enable  Auto                  Down   Enable  Enable long
0/5           Enable  Auto                  Down   Enable  Enable long
0/6           Enable  Auto                  Down   Enable  Enable long
0/7           Enable  Auto                  Down   Enable  Enable long
0/8           Enable  Auto                  Down   Enable  Enable long
0/9           Enable  Auto                  Down   Enable  Enable long
0/10          Enable  Auto                  Down   Enable  Enable long
0/11          Enable  Auto                  Down   Enable  Enable long
0/12          Enable  Auto                  Down   Enable  Enable long
0/13          Enable  Auto                  Down   Enable  Enable long
0/14          Enable  Auto                  Down   Enable  Enable long
0/15          Enable  Auto                  Down   Enable  Enable long
0/16          Enable  Auto                  Down   Enable  Enable long
0/17          Enable  Auto                  Down   Enable  Enable long
0/18          Enable  Auto                  Down   Enable  Enable long

--More-- or (q)uit


               Admin   Physical   Physical   Link   Link    LACP   Actor
Intf   Type    Mode    Mode       Status   Status  Trap    Mode   Timeout
------ ------ ------- ---------- ---------- ------ ------- ------ --------
0/19          Enable  Auto                  Down   Enable  Enable long
0/20          Enable  Auto                  Down   Enable  Enable long
0/21          Enable  Auto                  Down   Enable  Enable long
0/22          Enable  Auto                  Down   Enable  Enable long
0/23          Enable  Auto                  Down   Enable  Enable long
0/24          Enable  Auto                  Down   Enable  Enable long
0/25          Enable  Auto                  Down   Enable  Enable long
0/26          Enable  Auto                  Down   Enable  Enable long
0/27          Enable  Auto                  Down   Enable  Enable long
0/28          Enable  Auto                  Down   Enable  Enable long
0/29          Enable  Auto                  Down   Enable  Enable long
0/30          Enable  Auto                  Down   Enable  Enable long
0/31          Enable  Auto                  Down   Enable  Enable long
0/32          Enable  Auto                  Down   Enable  Enable long
0/33          Enable  Auto                  Down   Enable  Enable long
0/34          Enable  Auto                  Down   Enable  Enable long
0/35          Enable  Auto                  Down   Enable  Enable long
0/36          Enable  Auto                  Down   Enable  Enable long

--More-- or (q)uit


               Admin   Physical   Physical   Link   Link    LACP   Actor
Intf   Type    Mode    Mode       Status   Status  Trap    Mode   Timeout
------ ------ ------- ---------- ---------- ------ ------- ------ --------
0/37          Enable  Auto                  Down   Enable  Enable long
0/38          Enable  Auto                  Down   Enable  Enable long
0/39          Enable  Auto                  Down   Enable  Enable long
0/40          Enable  Auto                  Down   Enable  Enable long
0/41          Enable  Auto                  Down   Enable  Enable long
0/42          Enable  Auto                  Down   Enable  Enable long
0/43          Enable  Auto                  Down   Enable  Enable long
0/44          Enable  Auto                  Down   Enable  Enable long
0/45          Enable  Auto                  Down   Enable  Enable long
0/46          Enable  Auto                  Down   Enable  Enable long
0/47          Enable  Auto                  Down   Enable  Enable long
0/48          Enable  Auto                  Down   Enable  Enable long
0/49          Enable  10G Full   10G Full   Up     Enable  Enable long
0/50          Enable  10G Full              Down   Enable  Enable long
0/51          Enable  10G Full   10G Full   Up     Enable  Enable long
0/52          Enable  10G Full              Down   Enable  Enable long

(FASTPATH Routing) #conf

(FASTPATH Routing) (Interface 0/49)#no vlan pvid

(FASTPATH Routing) (Interface 0/49)#no vlan acceptframe

(FASTPATH Routing) (Interface 0/49)#no vlan tagging 100

(FASTPATH Routing) (Interface 0/49)#vlan participation auto 100

(FASTPATH Routing) (Interface 0/49)#no mtu

(FASTPATH Routing) (Interface 0/49)#
(FASTPATH Routing) #configure

(FASTPATH Routing) (Config)#interface 0/51

(FASTPATH Routing) (Interface 0/51)#vlan participation auto 100

(FASTPATH Routing) (Interface 0/51)#no vlan tagging 100

(FASTPATH Routing) (Interface 0/51)#no vlan pvid

(FASTPATH Routing) (Interface 0/51)#no vlan acceptframe

(FASTPATH Routing) (Interface 0/51)#no mtu

(FASTPATH Routing) (Interface 0/51)#
(FASTPATH Routing) #show running-config
!Current Configuration:
!
!System Description "Quanta LB4G, L.10.14.1, Linux 2.6.21.7"
!System Software Version "L.10.14.1"
!System Up Time          "0 days 15 hrs 10 mins 26 secs"
!Additional Packages     FASTPATH QOS,FASTPATH Multicast
!Current SNTP Synchronized Time: Not Synchronized
!
telnetcon timeout 160
network parms 10.0.1.2 255.255.255.0 10.0.1.1
vlan database
vlan 30,35,40,45,50,100
vlan name 30 "DMZ"
vlan name 35 "Wireless"
vlan name 40 "WirelessGuest"
vlan name 45 "SecurityCamera"
vlan name 50 "Management"
vlan name 100 "vSAN"
exit
configure
logging console 6
aaa authentication enable "enableList" enable
line console
exit

--More-- or (q)uit

line telnet
exit
line ssh
exit
spanning-tree configuration name "00-26-9E-02-59-6C"
!
interface 0/49
no auto-negotiate
exit
interface 0/50
no auto-negotiate
exit
interface 0/51
no auto-negotiate
exit
interface 0/52
no auto-negotiate
exit
router rip
exit

--More-- or (q)uit

router ospf
exit
exit

(FASTPATH Routing) #show mac-addr-table

      MAC Address        Interface  IfIndex     Status
-----------------------  ---------  -------  ------------
00:01:00:04:23:E5:6D:C7   0/1       1        Learned
00:01:00:0C:29:33:E6:1F   0/1       1        Learned
00:01:00:1B:21:8B:61:FF   0/1       1        Learned
00:01:00:1C:DF:54:2E:94   0/1       1        Learned
00:01:00:1E:67:84:C9:5A   0/1       1        Learned
00:01:00:1E:67:84:CE:D6   0/1       1        Learned
00:01:00:25:90:DD:68:9E   0/1       1        Learned
00:01:00:25:90:DD:6B:B8   0/1       1        Learned
00:01:00:26:9E:02:59:6C   3/1       53       Management
00:01:00:50:56:A7:51:45   0/1       1        Learned
00:01:00:50:56:A7:C4:0B   0/1       1        Learned
00:01:00:61:71:C5:CB:E8   0/1       1        Learned
00:01:00:BB:3A:AD:5F:DC   0/1       1        Learned
00:01:00:E0:36:EF:9F:AC   0/1       1        Learned
00:01:24:77:03:60:56:20   0/1       1        Learned
00:01:24:A4:3C:34:FC:06   0/1       1        Learned
00:01:44:19:B6:32:12:0E   0/1       1        Learned
00:01:44:19:B7:27:0E:04   0/1       1        Learned
00:01:44:19:B7:27:0E:06   0/1       1        Learned
00:01:6C:AD:F8:50:5F:1C   0/1       1        Learned
00:01:78:4B:87:72:70:34   0/1       1        Learned
--More-- or (q)uit


      MAC Address        Interface  IfIndex     Status
-----------------------  ---------  -------  ------------
00:01:7C:E9:D3:F1:9B:02   0/1       1        Learned
00:01:90:F1:AA:CB:54:CF   0/1       1        Learned
00:01:D0:50:99:27:68:D6   0/1       1        Learned
00:01:D4:4B:5E:FE:FF:EA   0/1       1        Learned
00:01:DC:9F:DB:82:2D:49   0/1       1        Learned

(FASTPATH Routing) #show interface 0/49


Packets Received Without Error................. 0
Packets Received With Error.................... 0
Broadcast Packets Received..................... 0
Packets Transmitted Without Errors............. 6849
Transmit Packet Errors......................... 0
Collision Frames............................... 0
Time Since Counters Last Cleared............... 0 day 15 hr 19 min 4 sec

(FASTPATH Routing) #show interface 0/51


Packets Received Without Error................. 0
Packets Received With Error.................... 0
Broadcast Packets Received..................... 0
Packets Transmitted Without Errors............. 7052
Transmit Packet Errors......................... 0
Collision Frames............................... 0
Time Since Counters Last Cleared............... 0 day 15 hr 19 min 16 sec

(FASTPATH Routing) #
Again, unplugging the two Brocade CNAs from the switch and then connecting them to each other with one fiber cable works and I get full connectivity between the two bare metal hosts with pinging and iperf working great between the two VMs. I wish this switch had more commands to query the transceivers directly for information.


EDIT: The fact that nothing is being received by either port is really odd. Looking at the monoprice 50/125 type fiber I'm using, the individual fiber strands are crossed properly (from TX to RX). The fiber is also obviously not to blame since I can pass traffic between the two Brocade CNAs.
 
Last edited:
E

Emulsifide

Active Member
Dec 1, 2014
212
93
28
h0tw1r3 said:
Confirmed. I am seeing the same issue. I'm confident it has nothing to do with the cables. Will have to wait until tomorrow as I have guests over tonight.
Click to expand...
Awesome...but not. I'm confident it's the switch as well. This really sucks, but hopefully we can figure it out. I'll keep tinkering and report back if I have any progress.
 
B

bds1904

Active Member
Aug 30, 2013
271
76
28
I use finisar optics on my lb4m, I wonder if those would make a difference. BrocaBrocade sfp+ in the nic and finisar in the switch?

Edit, just sasaw that's what you did. Mabie brocade on both sides or a dac?

I learned about the finisar optics from the eBay seller. They said that's what the datacenter that the switches came out of used because that's what quanta spec'ed. There is a pic of one in the lb4m thread.

Also, change participation to yes.
 
Last edited:
E

Emulsifide

Active Member
Dec 1, 2014
212
93
28
bds1904 said:
I use finisar optics on my lb4m, I wonder if those would make a difference. BrocaBrocade sfp+ in the nic and finisar in the switch?

Edit, just sasaw that's what you did. Mabie brocade on both sides or a dac?

I learned about the finisar optics from the eBay seller. They said that's what the datacenter that the switches came out of used because that's what quanta spec'ed. There is a pic of one in the lb4m thread.

Also, change participation to yes.
Click to expand...
Sorry, I didn't complete my statement. When I said "I just tried the Brocade transceivers I just got in the mail with the same results", this meant I placed the Brocade transceivers I got in the mail on the switch side (and I left the Brocade transceivers I already had on the Brocade 1020 CNA side as well). I don't have a DAC to try. Again, link looks good on both the host and switch side, but still no ping, no MACs showing up, and no received packets on the switch interfaces.

Here's the options for participation:

Code: 
(FASTPATH Routing) (Interface 0/49)#vlan participation ?

auto                     Set participation to auto.
exclude                  Set participation to exclude.
include                  Set participation to include.
Doing a "set participation include 1" for allowing the port to use VLAN 1 does nothing because it's already assumed through auto that VLAN 1 is included.
 
h0tw1r3

h0tw1r3

Member
Oct 30, 2014
44
21
8
United States
zaplabs.com
Tcpdump shows nothing on the host side.

I happen to have some new sfp+ modules coming tomorrow.

I have a sneaking suspicion that the vxworks image is buggy/not latest available resulting in transceiver incompatibilities.
 
Last edited:
E

Emulsifide

Active Member
Dec 1, 2014
212
93
28
h0tw1r3 said:
I'm 99% positive the firmware is not correct for this device.
While in the diag app, the SFP+ lights change on the wrong port and PortStat up/down status is wrong.
Broadcom Command monitor shows LB4M as the platform.
It's almost as if this image should have been on the LB4M with u-boot.

I'm hesitant to flash openflow / indigo because I have to upgrade u-boot also, which I don't think is backwards compatible.
Click to expand...
I think you're absolutely right. Did you notice this line at boot when it gets to the image:

Code: 
## Booting kernel from Legacy Image at 08000074 ...
   Image Name:   FASTPATH System for lb4m
I'm also hesitant to do openflow for the same reason. I'd rather return the switch and keep looking for another. There's no chance of us being able to find the proper image for this switch on the internet. Then again, I wonder if the Indigo developers have a copy hosted somewhere. Hmmm...
 
You must log in or register to reply here.
Top Bottom