Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[itronin]

@Paul Mew

so dual-mode is brocade specific way of having tagged and untagged vlans on the same port. You want 1/1/1 to be dual-mode allowing it to pass untagged vlan1 and tagged vlan40. Basically follow example 1 below in the link below and substitute vlan 40 and 1/1/1 in.

Commscope Technical Content Portal

docs.commscope.com

@Paul Mew yep. looking it up pfsense can handle untagged traffic and tagged traffic as configured and I was wrong about whether it could or could not. Though as to it being recommended there are some rather emphatic arguments about not doing it that way as well as not using vlan 1.

@Vesalius has provided the deets on how to configure this in the 6450.

 

[Paul Mew]

Thanks so much for the detailed reply.

The aim is eventually to put the quad NIC back in (...it's in the post) but I guessed that starting with just the one physical connection would keep it as simple as possible.

Yes, I believe I do just want to use it as a switch and let pfsense do the routing. It seems as though I stumbled naively through with the HP2910 as it seems to be closer to the consumer stuff I've worked with before and possibly it's more intuitive to a rookie like me.....

I'll take some time to digest what you've said and have an experiment.......

One point I'm still in the dark with though is the application of the "Router Interface" selection in the VLAN setup? I still haven't a clue as to what value, if any, to make when setting up the VLAN.

Cheers,

Paul

 

[Paul Mew]

@Paul Mew

so dual-mode is brocade specific way of having tagged and untagged vlans on the same port. You want 1/1/1 to be dual-mode allowing it to pass untagged vlan1 and tagged vlan40. Basically follow example 1 below in the link below and substitute vlan 40 and 1/1/1 in.

Commscope Technical Content Portal

docs.commscope.com

Thanks for the lead! I'm heading right over there.

 

[itronin]

edit - grammar clarification

One point I'm still in the dark with though is the application of the "Router Interface" selection in the VLAN setup? I still haven't a clue as to what value, if any, to make when setting up the VLAN.

Only if you were going to use the 6450 to handle L3 routing for your inter-vlan traffic.
You will be routing inter-vlan traffic through pfsense.

 

[Vesalius]

@Paul Mew yep. looking it up pfsense can handle untagged traffic and tagged traffic as configured and I was wrong about whether it could or could not. Though as to it being recommended there are some rather emphatic arguments about not doing it that way as well as not using vlan 1.

@Vesalius has provided the deets on how to configure this in the 6450.

I would second being cautious of vlan 1. @Paul Mew - you can always add a different vlan on the 6450 for your LAN (let’s say 10) later and then just changing port 1/1/1 with dual-mode 10 will get the pfSense side of that equation sorted.

 

[klui]

Dual mode is only applicable to older ICX 6000-series switches and older FW on ICX 7000 series. If you get a 7xxx and use version 8.0.80 and newer FW there is no more dual mode.

 

[kpfleming]

Dual mode is only applicable to older ICX 6000-series switches and older FW on ICX 7000 series. If you get a 7xxx and use version 8.0.80 and newer FW there is no more dual mode.

Just to be clear: the 'dual mode' is no longer required to support tagged and untagged traffic on the same port. The functionality still exists, it just doesn't have that name.

 

[klui]

That's just being pendantic. Of course the ability exists to tagged and untagged VLANs on a port. The functionality is just using industry standard syntax with a caveat.

 

[kpfleming]

Indeed it is being pedantic, but given the number of relatively-newbie networking people in this thread, I thought it might be useful to be clear that the functionality still exists, it's just named differently

 

[itronin]

LOL. my brain doesn't work right. my quick glance at @klui 's post saw

Duel of Death Mode.

which seems appropriate when being pedantic.

 

[MelnorMelvin]

One more question. I am needing to pick up 2x ICX-FAN10-E for my ICX6610-48P-PE, and was hoping not to spend a fortune on fans.

Pretty much as cheap as I can find for an item actually tagged as ICX-FAN10-E is ~$49.50, some with a P/N of 80-1008308-02 on the factory box.

However, I did come across an item which looks physically identical with part number 60-1002274-03 (sticker on the bottom of the fans) at a much more reasonable ~$14.99, with bulk discounts (qty 2 at $13.49/ea).

Does anyone know if these two items are actually the same?

Good news is that I have confirmed these fan modules are the same as the ICX-FAN10-E's.

If you could remember to ping, ping us (or me) when you get the fans and let us know. I am in need of 1 or 2 as well.

PING.

 

[fohdeesha]

Good news is that I have confirmed these fan modules are the same as the ICX-FAN10-E's.



PING.

can you post the output of "dm fan-speed" and "show chassis | inc Fan"

 

[MelnorMelvin]

can you post the output of "dm fan-speed" and "show chassis | inc Fan"

Okay. Fan 1 is the one I bought from eBay for $15, and Fan 2 is one of the original ones that came with the switch.

ICX6610-48P-Router#dm fan-speed
FRU 1 Fan 1 Speed at 5056 RPM.
FRU 1 Fan 2 Speed at 5465 RPM.
FRU 2 Fan 1 Speed at 5465 RPM.
FRU 2 Fan 2 Speed at 5869 RPM.

ICX6610-48P-Router#show chassis | inc Fan
Power supply 1 Fan Air Flow Direction: Front to Back
Power supply 2 Fan Air Flow Direction: Front to Back
Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2
Fan controlled temperature: 55.0 deg-C
Fan speed switching temperature thresholds:
Fan 1 Air Flow Direction: Front to Back
Fan 2 Air Flow Direction: Front to Back

 

[BobTB]

Yes, you will need a VE in both VLANs. One will have your ISP-assigned endpoint address from the /30 subnet, and the other will have one of the addresses from the /27 (which will be the gateway address for the other devices on that VLAN).

I need to set the route 0.0.0.0/0 22.22.33.21 (isp assigned upstream gateway) on the /30 VLAN, with the VE having IP 22.22.33.22.

hm, I think I am missing a static route from the /27 vlan to the /30 to make the /27 be able to reach internet.
ip route 0.0.0.0/0 ve30 (where ve30 is the VE in vlan having /30 subnet)

Will this work, or is it better to go with the IP assigned to that ve30 (22.22.33.22)? I read somewhere this will cause very large ARP cache?

I will actually have routers connected to the /27 VLAN ports, to NAT the IPs I got from the ISP.

 

[BobTB]

I need to set the route 0.0.0.0/0 22.22.33.21 (isp assigned upstream gateway) on the /30 VLAN, with the VE having IP 22.22.33.22.
I will actually have routers connected to the /27 VLAN ports, to NAT the IPs I got from the ISP.

Actually, can I just use the VLAN in which I have the /30 from ISP assigned to VE30 (22.22.33.22) if I just create another VE27 in the same VLAN with one of the assigned IPs from the /27 (and set its default route to the VE30) and use this VE27 IP as default gateway in connected PFSense routers, removing the need to have two vlans?

 

[JoJoMan]

I'm having a similar issue as @Paul Mew I believe.

Everything is working fine on the default vlan. I only have 1 upstream connection to the router (1/1/1) and seems when I try to add it to vlan 2, it takes out everything on vlan 1 :/

with default vlan:

#show vlan
Total PORT-VLAN entries: 1
Maximum PORT-VLAN entries: 64

Legend: [Stk=Stack-Id, S=Slot]

PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree Off
Untagged Ports: (U1/M1)   1   2   3   4   5   6   7   8   9  10  11  12
Untagged Ports: (U1/M1)  13  14  15  16  17  18  19  20  21  22  23  24
Untagged Ports: (U1/M1)  25  26  27  28  29  30  31  32  33  34  35  36
Untagged Ports: (U1/M1)  37  38  39  40  41  42  43  44  45  46  47  48

Untagged Ports: (U1/M2)   1   2   3   4
   Tagged Ports: None
   Uplink Ports: None
DualMode Ports: None
Mac-Vlan Ports: None
     Monitoring: Disabled

I add a new vlan

#vlan 2
#tagged ethernet 1/1/1

and after that I lose connection to the switch as it seems 1/1/1 gets removed from vlan 1 and only gets added to vlan 2.

I'm probably an idiot, but IDK anything about networking

I was trying to follow this guide, which does something similar (i think)

VLAN configuration on Ruckus ICX switches for Ruckus Wireless WLAN's | Knowledge Base | Ruckus Wireless Support

support.ruckuswireless.com

 

[Necrotyr]

I'm having a similar issue as @Paul Mew I believe.

Everything is working fine on the default vlan. I only have 1 upstream connection to the router (1/1/1) and seems when I try to add it to vlan 2, it takes out everything on vlan 1 :/

with default vlan:

#show vlan
Total PORT-VLAN entries: 1
Maximum PORT-VLAN entries: 64

Legend: [Stk=Stack-Id, S=Slot]

PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree Off
Untagged Ports: (U1/M1)   1   2   3   4   5   6   7   8   9  10  11  12
Untagged Ports: (U1/M1)  13  14  15  16  17  18  19  20  21  22  23  24
Untagged Ports: (U1/M1)  25  26  27  28  29  30  31  32  33  34  35  36
Untagged Ports: (U1/M1)  37  38  39  40  41  42  43  44  45  46  47  48

Untagged Ports: (U1/M2)   1   2   3   4
   Tagged Ports: None
   Uplink Ports: None
DualMode Ports: None
Mac-Vlan Ports: None
     Monitoring: Disabled

I add a new vlan

#vlan 2
#tagged ethernet 1/1/1

and after that I lose connection to the switch as it seems 1/1/1 gets removed from vlan 1 and only gets added to vlan 2.

I'm probably an idiot, but IDK anything about networking

I was trying to follow this guide, which does something similar (i think)

VLAN configuration on Ruckus ICX switches for Ruckus Wireless WLAN's | Knowledge Base | Ruckus Wireless Support

support.ruckuswireless.com

If you're running 8.0.30 you need to configure dual-mode on E1/1/1.

Dual-mode VLAN ports

docs.ruckuswireless.com

 

[kpfleming]

Actually, can I just use the VLAN in which I have the /30 from ISP assigned to VE30 (22.22.33.22) if I just create another VE27 in the same VLAN with one of the assigned IPs from the /27 (and set its default route to the VE30) and use this VE27 IP as default gateway in connected PFSense routers, removing the need to have two vlans?

I believe you are overthinking this a bit

Routes are not specific to VLANs or VEs (generally). They are global entries in the routing table on the device. When you create a VE in VLAN30, and a VE in VLAN27, and put IP addresses/subnets on those VEs, then the ICX has routes to those subnets and can route traffic between them. The only additional step for devices on those subnets to talk to each other is for *them* to be configured to know that they should use the ICX as their gateway (either default - for all traffic, or explicit for those subnets).

In addition to the internal routes between the VLANs, you'll need a default route on the ICX to tell it how to route traffic to external destinations. That should be a 0.0.0.0 route to the ISP's provided gateway address, which will be an address within the /30 subnet. The ICX will know that it needs to send traffic out through that VE/VLAN due to the gateway address.

That should be it: any device on the /27 subnet with a default gateway set to the ICX will then be able to send traffic to external destinations.

And no, you cannot create two VEs in the same VLAN. In a non-VLAN world, you'd be using two physical interfaces on a router, connected to separate physical subnets, so you'll be emulating that using VLANs.

 

[BobTB]

I believe you are overthinking this a bit

Thank you, will read it a few times and try to implement it that way. Was watching this video

and it seemed as if setting a route for a VE is a thing somehow.

 

[Nystral]

Hello all,

I bought an ebay ICX7250-48 running SW: Version 08.0.30hT213 and running into issues getting it to 8095

I'm following the guide fohdeesha posted on page 1

I run through and update update of uboot with no issues.

factory set-default completes quickly

update_primary returns the below:

ICX7250-Boot> update_primary
Updating PRIMARY image over TFTP
Using bcmiproc_eth-0 device
TFTP from server 192.168.1.12; our IP address is 192.168.1.2
Filename 'ICX7xxx/SPR08090mc.bin'.
Load address: 0x1000000
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         ###########
done
Bytes transferred = 33554432 (2000000 hex)
img_sz=0x2000000 block_count = 0x20

NAND erase.spread: device 0 offset 0x0, size 0x2000000
Erasing at 0x1f00000 -- 100% complete.
OK

NAND write: device 0 offset 0x0, size 0x2000000
................................ 33554432 bytes written: OK
TFTP to Flash Done.

As far as I can tell everything looks good

however boot_primary returns the below:

resetting ...


Brocade Bootloader: 10.1.06T215 (Dec 14 2015 - 13:10:57)

Validate Shmoo parameters stored in flash ..... OK
Restoring Shmoo parameters from flash .....
Running simple memory test ..... OK

ICX7250-48 Copper (Non-POE), PVT1
SYS CPLD VER: 0x6, Released Ver: 0

Enter 'b' to stop at boot monitor:  0
bootdelay: ===
Booting image from Primary

NAND read: device 0 offset 0x0, size 0x2000000
................................ 33554432 bytes read: OK
Wrong Image Format for bootm command
ERROR: can't get kernel image!
could not boot from primary, no valid image; trying to boot from secondary
BOOTING image from Secondary

NAND read: device 0 offset 0x4000000, size 0x2000000
................................................................ 33554432 bytes read: OK
## Booting kernel from Legacy Image at 70000200 ...
   Image Name:   Image
ARM Linux Kernel Image (uncompressed)
   Data Size:    31678084 Bytes =    Load Address: 61008000
   Entry Point:  61008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK

Starting kernel ...

MD5 sum on the downloaded 08090mc.bin is ea452ea2a066c27051bcb342ef67b13c and matches what in on the TFTP server and my downloaded copy fresh as of 2 hours ago.

show version tells me my uboot is right

Compressed Boot-Monitor Image size = 786944, Version:10.1.06T215 (spz10118  )

Boot Environment

ICX7250-Boot> show_image
===========================
Primary  :
NAND read: device 0 offset 0x0, size 0x200
 512 bytes read: OK
SPR08090mc(33554432)
Secondary:
NAND read: device 0 offset 0x4000000, size 0x200
 512 bytes read: OK
SPR08030h(31678660)
===========================

I'm sorry if this has been answered up thread but I can't seem to find the answer in my searching.