Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[tls]

ICX6450 to ICX7150 - multi-gig SFP+

Are the Mikrotik S+RJ10 multi-gig SFP+ modules expected to work in an ICX6450? In a 7150 with up to date FastIron? At speeds other than 1G and 10G?

I just bought a pair and tried to deploy them between a 6450-24P and 7150-C12P on a roughly 100' cat6 run through 2 patch panels across my house - I'd expect this run to be 2.5G or maybe 5G capable but certainly not 10G - and they won't link up.

Interestingly, if I pull the multi-gig module from the 7150 end and replace with a 1G module, then the link does eventually come up, even though I do not have speed-duplex set to 1000. But with a multi-gig module at both ends, nada.

On the 7150, I see something very odd when I put one of the modules into a slot and show media ethernet:

edge-office>show media ethernet 1/3/2
Port   1/3/2: Type  : 1GE M-SX(SFP)
         Vendor: OTHERSik             Version: 2.16
         Part# : S+RJ10               Serial#: HCK01PA361Q

"OTHERSik"?

I did notice I had to insert the module very firmly or the slot just showed as EMPTY. But it's definitely seated properly now, and the other identical module I have also shows as an OTHERSik.

 

[Craig Curtin]

Mellanox COnnect CX3 to 6610

Guys, my first foray into 10Gbe (and higher) - i have followed along with the tutorial and got a couple of 6610 up and running fine. I want to now fitout a couple of machines with 10Gbe connections and am hoping to use the breakout ports on the rear (1/2/2-1/2/5 and 1/2/7 - 1/2/10) for 10GBe connections to a couple of servers in y home lab.

I am interested in the Mellanox CX3 (OEM) cards avaiable on ebay in australia as they are relatively cheap - i am just wondering though - is the physical slot just a SFP+ slot on these cards - so i can get one of the QSFP+ to SFP+ 4 way breakout cables - to allow me to connect 4 of the servers (at 10Gb) into one of the breakout cables - or is it a different connector on the Mellanox ?

Craig

 

[mach3.2]

i can get one of the QSFP+ to SFP+ 4 way breakout cables - to allow me to connect 4 of the servers (at 10Gb) into one of the breakout cables - or is it a different connector on the Mellanox ?

This works, but you'll have to use the 40GbE ports that can breakout into 4* 10GbE connections.

On the ConnectX 3 end, get the SFP+ version(CX311/CX312) instead of the QSFP+ 40GbE version(CX353/CX354).

If you have the 40GbE ConnectX 3 cards, you'll need QSA adapters and those cost around 20 bucks a pop.

 

[clifford64]

Maybe I am just not understanding something, but what is the proper way to enable multicast routing on a 6450? It is my understanding that if I have the below two commands, this enables multicast routing.
ip multicast passive
ip pimsm-snooping

It is my understanding that it will perform listing for multicast requests and keep a cache of which multicast groups there are on different interfaces. So if a multicast is sent to a specific group, it will be forwarded to each interface within that group even if that interface is on a separate VLAN (as long as you have intervlan routing setup).

So then in theory, if vlan 5 and vlan 17 both have a common group, a multicast packet sent from vlan 5 to that group address would arrive on the devices in vlan 17 that are in that group right?

I am essentially trying to get UPNP requests to hit my upstream router from an internal VLAN.

 

[Craig Curtin]

This works, but you'll have to use the 40GbE ports that can breakout into 4* 10GbE connections.

On the ConnectX 3 end, get the SFP+ version(CX311/CX312) instead of the QSFP+ 40GbE version(CX353/CX354).

If you have the 40GbE ConnectX 3 cards, you'll need QSA adapters and those cost around 20 bucks a pop.

Thanks very much for the detailed response - i am now officially on the hunt !

Craig

 

[Rossatron]

So, I have the icx 6610, and decided to try the 40g ports (not the breakout, they are in use but not relevant to my question) so I bought an adaptor from startech that has a 40gb qsfp port. I’ve removed 1/2/6 from stack (stack was disabled anyways) added it to my home vlan. Plugged the two together with arista qsfp modules and a fancy fibre cable. Got a link up at 40gbs. But no network traffic. Couldn’t get a dhcp IP address. Check the dhcp server logs and it was being asked for an ip and it was sending it but it never gets to the device connected to 1/2/6. I manually gave the device on 1/2/6 an ip and I could then ping any device on my network. But that’s it, literally all I can do is ping. I can’t access scp, ssh, html. Literally nothing appears to get through. It’s on the same vlan as all other devices so I’m at a loss. It’s definitely not showing as being a stacking port. But I can’t for the life of me fathom where to start troubleshooting. I’ve tried brocade qsfp transceivers, a second fibre cable, a different nic all with the same results. I even tried 1/2/1 and had exactly the same results. Only time I’ve ever been able to ping but no network access has been due to vlan issues but I’ve triple checked it’s in the vlan. Have tried tagging the port instead of untagged and I can't even ping it then. I’ve seen a few others have had a similar issue from reading this thread but no actual solutions have been posted, and yes I did try rebooting everything, crazily enough rebooting the switch seems to fix a few issues! So yeah, anyone with any ideas?

After further diagnostics, using tagged lan ensures the client is on the arp table for the router. doing a "ping google.com" obtains an IP address from the dns but it can't actually ping the ip its given.

Conversely removing the tag and adding it as untagged, I am able too ping local computers, and can obtain ip addresses from dns server. The client still shows on arp table though albeit with the wrong hostname, (Same hostname as the first windows based client I tried the network card in) But the ping actually goes out to the service (i.e. google, Facebook)

Traceroute doesn't work in either case.

So from all of the above from what I can tell I may be looking at a faulty NIC as vlan works fine with the two breakout ports.

 

[pypypy]

I have a 6450-48p with a dead POE module. Decided to pull the module completely because I had issues with ports randomly going down and back up again... After doing so, the fan speed is stuck at 100%. How do I remove the POE firmware so the switch thinks its a non-poe?

 

[nextrack68]

In over my head with a 6450p trying to configure VLANS. The switch works great with wired devices including two TP Link EAP245 WAP powered by POE through the switch.

I have a pfSense router plugged into port 1 of the 6450p and a WAP into port 6 & 7. I am running TPLink Omada (as a docker) to control the WAP. So far so good as long as I am not trying to use a VLAN on the WAP.

I am trying to have a VLAN 20 & 30. Some devices will be wired on an individual 6450 port and other will be wireless. I have VLANS 20 & 30 set up in Omada with a corresponding SSID for each. My phone can see each of these networks and join them but cannot receive internet. I also have the VLANs set up in pfSense.

Finally, I am trying to set up the correct tagged/untagged/dual VLAN settings on the 6450p correctly. Every combination I try seems to break things. I have searched and searched and cannot seem to find an explanation I can understand (I am about caveman status).

For example: Port 22 on the 6450P goes to a TV that I want on VLAN 30. Every time I set the VLAN on Port 22 I lose internet access. It's like the switch isn't passing the VLAN on to pfSense.

Another example: I want two SSID's for IoT and Kids, VLANS 20 & 30. My phone sees the SSIDs and can connect, but does not have internet access.

Thank you

 

[Craig Curtin]

In over my head with a 6450p trying to configure VLANS. The switch works great with wired devices including two TP Link EAP245 WAP powered by POE through the switch.

I have a pfSense router plugged into port 1 of the 6450p and a WAP into port 6 & 7. I am running TPLink Omada (as a docker) to control the WAP. So far so good as long as I am not trying to use a VLAN on the WAP.

I am trying to have a VLAN 20 & 30. Some devices will be wired on an individual 6450 port and other will be wireless. I have VLANS 20 & 30 set up in Omada with a corresponding SSID for each. My phone can see each of these networks and join them but cannot receive internet. I also have the VLANs set up in pfSense.

Finally, I am trying to set up the correct tagged/untagged/dual VLAN settings on the 6450p correctly. Every combination I try seems to break things. I have searched and searched and cannot seem to find an explanation I can understand (I am about caveman status).

For example: Port 22 on the 6450P goes to a TV that I want on VLAN 30. Every time I set the VLAN on Port 22 I lose internet access. It's like the switch isn't passing the VLAN on to pfSense.

Another example: I want two SSID's for IoT and Kids, VLANS 20 & 30. My phone sees the SSIDs and can connect, but does not have internet access.

Thank you

Post up what you have in terms of the switch config

Have you set a default gateway on the switch - which should point to your PFSense

Presumably you have a trunk setup to the Pfsense box to carry both of the VLANs ?

Also show some screenshots of what you have in Pfsense

Essentially you should have a tagged port on the 6450 in both VLAN 20 and 30 and this should connect to PFsense which also has that port defined as Tagged with the two VLANs

Craig

Craig

 

[Vesalius]

In over my head with a 6450p trying to configure VLANS. The switch works great with wired devices including two TP Link EAP245 WAP powered by POE through the switch.

I have a pfSense router plugged into port 1 of the 6450p and a WAP into port 6 & 7. I am running TPLink Omada (as a docker) to control the WAP. So far so good as long as I am not trying to use a VLAN on the WAP.

I am trying to have a VLAN 20 & 30. Some devices will be wired on an individual 6450 port and other will be wireless. I have VLANS 20 & 30 set up in Omada with a corresponding SSID for each. My phone can see each of these networks and join them but cannot receive internet. I also have the VLANs set up in pfSense.

Finally, I am trying to set up the correct tagged/untagged/dual VLAN settings on the 6450p correctly. Every combination I try seems to break things. I have searched and searched and cannot seem to find an explanation I can understand (I am about caveman status).

For example: Port 22 on the 6450P goes to a TV that I want on VLAN 30. Every time I set the VLAN on Port 22 I lose internet access. It's like the switch isn't passing the VLAN on to pfSense.

Another example: I want two SSID's for IoT and Kids, VLANS 20 & 30. My phone sees the SSIDs and can connect, but does not have internet access.

Thank you

Posting your switch config as @Craig Curtin suggested will allow for better advice. For endpoint devices that do not natively support vlans (most things other than a computer or enterprise switch/router/AP) that plug into the switch you will need that switch port to be untagged in whatever vlan you want the endpoint to be in.

Generally though to take one specific item from your questions as an example.
For Example: SwitchPort 22 should be untagged Vlan30, which will tag network packets from your TV with vlan30 in the switch. SwitchPort 1 needs to be tagged vlan30 (otherwise it won't allow packets tagged by pfsense with vlan30 or switchport 22 with vlan30 to pass) AND your pfsense needs to have a vlan30 setup using the interface which ultimately plugs into switchport 1 as parent. Pfsense needs specific firewall rules allowing access to WAN.

 

[sputnik13]

I have an ICX6450 48P, pondering mounting the switch at the back of my rack so I don't have to get a patch panel and worried about cooling.

This is an enclosed rack with solid walls to reduce noise with an inlet in the front and exhaust in the rear for airflow.

Has anyone had success/failure with heat management in this configuration? Would reversing the fans to blow in through the back of the switch help?

 

[jayb998]

ICX6450-24P... what fans is everyone switching to? I've seen Noctua mentioned and also Sunon - which ones are the preferred models I should be looking for? I need something quiet but it doesn't have to be silent. Expecting POE usage approx ~75-100W in total.

 

[Craig Curtin]

Thanks very much for the detailed response - i am now officially on the hunt !

Craig

These Mellanox cards with the SFP+ (rather than QSFP) connectors are few and far between here in Australia ! And i can not find any reasonably priced adapters for the conversion of the QSFP to SFP+ slot - so looks like i will be trying to source the Intel Dual port cards

Craig

 

[Craig Curtin]

Still having weird problems in my home lab with 6610s (tried two of them now - one POE and one non POE) - both flashed as per the excellent doc from Fodeesha to to the latest etc.

I am using dual port cards - primarily Intel 520-da2 and 540T (RJ45)

The cards with the SFP+ connectors i have tried to the 1/3/x slots and also to a FC.COM QSFP to SFP+ (1-4) breakout - i have 3 of these cables and have tried them all.

I have Dell SFP_ to RJ45 transceivers for the 1/3/x slots and have tried the 540T to those as well.

I am using these cards in Compaq/HP 8300 and Dell Optiplex 7050

These are running under ESXI 7.03

I am finding any changes on the switch ports that i make that are connected to one of the cards will invariably disable the port at the ESXi end and the only resolution appears to be a cold restart of the box - it does not matter which combination of ports/transceivers/breakouts - it is a fairly consistent problem

Something as simple as adding a port to a VLAN appears to be enough to trigger the problem

I have done an excerpt here of the config

Any Ideas ?

Craig





SSH@CURTO-6610#show run
Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
module 1 icx6610-48-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack disable
!
!
!
lag ToCisco dynamic id 1
ports ethernet 1/1/37 to 1/1/38
primary-port 1/1/37
deploy
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 100 name data-100 by port
tagged ethe 1/1/1 to 1/1/3 ethe 1/1/37 to 1/1/38 ethe 1/2/2 ethe 1/2/4 ethe 1/2/7 to 1/2/9 ethe 1/3/1 to 1/3/3 ethe 1/3/5
router-interface ve 100
!
vlan 101 name iot-101 by port
tagged ethe 1/1/37 to 1/1/38 ethe 1/2/2 ethe 1/2/4 ethe 1/2/7 to 1/2/9 ethe 1/3/1
!
vlan 102 name cryto-102 by port
tagged ethe 1/1/1 to 1/1/3 ethe 1/2/2 ethe 1/2/4 ethe 1/3/5
!
vlan 200 name storage-200 by port
tagged ethe 1/1/1 to 1/1/3 ethe 1/1/37 to 1/1/38 ethe 1/2/3 ethe 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/8
!
vlan 202 name prosis-202 by port
tagged ethe 1/1/1 to 1/1/4 ethe 1/1/37 to 1/1/38 ethe 1/2/2 ethe 1/2/4 ethe 1/2/7 to 1/2/9 ethe 1/3/1 to 1/3/3 ethe 1/3/5
router-interface ve 202
!
!
!
!
!
aaa authentication web-server default local
aaa authentication login default local
jumbo
enable aaa console
hostname CURTO-6610
ip dhcp-client disable
ip dns server-address 172.16.100.1
ip route 0.0.0.0/0 172.16.100.1
!
no telnet server
username root password .....
!
!
clock summer-time
clock timezone gmt GMT+10
!
!
ntp
disable serve
server 216.239.35.0
server 216.239.35.4
server 172.16.100.1
!
!
no web-management http
!
!
router rip
!
!
!
!
!
!
!
interface ethernet 1/1/1
port-name ESXI-Host1 - Onboard Management port
!
interface ethernet 1/1/2
port-name ESXI-Host2 - Onboard Management port
!
interface ethernet 1/1/3
port-name ESXI-Host3 - Onboard Management port
!
interface ethernet 1/1/4
port-name o7050-ESXI-2 - Onboard Management port
!
interface ethernet 1/2/2
port-name o7050-Host-1-I520-Port-1
!
interface ethernet 1/2/3
port-name o7050-Host-1-I520-Port-2
!
interface ethernet 1/2/4
port-name o7050-Host-2-I520-Port-1
!
interface ethernet 1/2/5
port-name o7050-Host-2-I520-Port-2
!
interface ethernet 1/2/7
port-name Host-1-520s
!
interface ethernet 1/2/8
port-name Host-2-520s
!
interface ethernet 1/2/9
port-name Host-3-520s
!
interface ethernet 1/3/1
port-name Host-3-540T-Port2
speed-duplex 10G-full
!
interface ethernet 1/3/2
port-name o7050-I540-Port2-Storage
speed-duplex 10G-full
!
interface ethernet 1/3/3
port-name Host-1-540T-Port 2
speed-duplex 10G-full
!
interface ethernet 1/3/4
speed-duplex 10G-full
!
interface ethernet 1/3/5
port-name o7050-Host-1-Port-2-Storage
speed-duplex 10G-full
!
interface ethernet 1/3/6
speed-duplex 10G-full
!
interface ethernet 1/3/7
port-name Mediastore-Fibre
speed-duplex 10G-full
!
interface ethernet 1/3/8
port-name Mediabackup-OMV-Fibre
speed-duplex 10G-full
!
interface ve 1
!
interface ve 100
ip address 172.16.100.254 255.255.255.0
!
interface ve 202
ip address 192.168.202.254 255.255.255.0
!
!
!
!
!
!
!
!
!
end

SSH@CURTO-6610#

 

[Craig Curtin]

Still having weird problems in my home lab with 6610s (tried two of them now - one POE and one non POE) - both flashed as per the excellent doc from Fodeesha to to the latest etc.

I am using dual port cards - primarily Intel 520-da2 and 540T (RJ45)

The cards with the SFP+ connectors i have tried to the 1/3/x slots and also to a FC.COM QSFP to SFP+ (1-4) breakout - i have 3 of these cables and have tried them all.

I have Dell SFP_ to RJ45 transceivers for the 1/3/x slots and have tried the 540T to those as well.

I am using these cards in Compaq/HP 8300 and Dell Optiplex 7050

These are running under ESXI 7.03

I am finding any changes on the switch ports that i make that are connected to one of the cards will invariably disable the port at the ESXi end and the only resolution appears to be a cold restart of the box - it does not matter which combination of ports/transceivers/breakouts - it is a fairly consistent problem

Something as simple as adding a port to a VLAN appears to be enough to trigger the problem

I have done an excerpt here of the config

Any Ideas ?

Craig

View attachment 25131

View attachment 25132

SSH@CURTO-6610#show run
Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
module 1 icx6610-48-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack disable
!
!
!
lag ToCisco dynamic id 1
ports ethernet 1/1/37 to 1/1/38
primary-port 1/1/37
deploy
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 100 name data-100 by port
tagged ethe 1/1/1 to 1/1/3 ethe 1/1/37 to 1/1/38 ethe 1/2/2 ethe 1/2/4 ethe 1/2/7 to 1/2/9 ethe 1/3/1 to 1/3/3 ethe 1/3/5
router-interface ve 100
!
vlan 101 name iot-101 by port
tagged ethe 1/1/37 to 1/1/38 ethe 1/2/2 ethe 1/2/4 ethe 1/2/7 to 1/2/9 ethe 1/3/1
!
vlan 102 name cryto-102 by port
tagged ethe 1/1/1 to 1/1/3 ethe 1/2/2 ethe 1/2/4 ethe 1/3/5
!
vlan 200 name storage-200 by port
tagged ethe 1/1/1 to 1/1/3 ethe 1/1/37 to 1/1/38 ethe 1/2/3 ethe 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/8
!
vlan 202 name prosis-202 by port
tagged ethe 1/1/1 to 1/1/4 ethe 1/1/37 to 1/1/38 ethe 1/2/2 ethe 1/2/4 ethe 1/2/7 to 1/2/9 ethe 1/3/1 to 1/3/3 ethe 1/3/5
router-interface ve 202
!
!
!
!
!
aaa authentication web-server default local
aaa authentication login default local
jumbo
enable aaa console
hostname CURTO-6610
ip dhcp-client disable
ip dns server-address 172.16.100.1
ip route 0.0.0.0/0 172.16.100.1
!
no telnet server
username root password .....
!
!
clock summer-time
clock timezone gmt GMT+10
!
!
ntp
disable serve
server 216.239.35.0
server 216.239.35.4
server 172.16.100.1
!
!
no web-management http
!
!
router rip
!
!
!
!
!
!
!
interface ethernet 1/1/1
port-name ESXI-Host1 - Onboard Management port
!
interface ethernet 1/1/2
port-name ESXI-Host2 - Onboard Management port
!
interface ethernet 1/1/3
port-name ESXI-Host3 - Onboard Management port
!
interface ethernet 1/1/4
port-name o7050-ESXI-2 - Onboard Management port
!
interface ethernet 1/2/2
port-name o7050-Host-1-I520-Port-1
!
interface ethernet 1/2/3
port-name o7050-Host-1-I520-Port-2
!
interface ethernet 1/2/4
port-name o7050-Host-2-I520-Port-1
!
interface ethernet 1/2/5
port-name o7050-Host-2-I520-Port-2
!
interface ethernet 1/2/7
port-name Host-1-520s
!
interface ethernet 1/2/8
port-name Host-2-520s
!
interface ethernet 1/2/9
port-name Host-3-520s
!
interface ethernet 1/3/1
port-name Host-3-540T-Port2
speed-duplex 10G-full
!
interface ethernet 1/3/2
port-name o7050-I540-Port2-Storage
speed-duplex 10G-full
!
interface ethernet 1/3/3
port-name Host-1-540T-Port 2
speed-duplex 10G-full
!
interface ethernet 1/3/4
speed-duplex 10G-full
!
interface ethernet 1/3/5
port-name o7050-Host-1-Port-2-Storage
speed-duplex 10G-full
!
interface ethernet 1/3/6
speed-duplex 10G-full
!
interface ethernet 1/3/7
port-name Mediastore-Fibre
speed-duplex 10G-full
!
interface ethernet 1/3/8
port-name Mediabackup-OMV-Fibre
speed-duplex 10G-full
!
interface ve 1
!
interface ve 100
ip address 172.16.100.254 255.255.255.0
!
interface ve 202
ip address 192.168.202.254 255.255.255.0
!
!
!
!
!
!
!
!
!
end

SSH@CURTO-6610#

An update to this

I have updated the drivers on the ESXi server (ixbgen) to the latest available from Vmware and this has made no difference - apparently these drivers are also meant to update the firmware on the cards when they run if required

I think i have partly solved the issue - it looks like i have a batch of defective cables from FS.COM (i purchase 3 of the breakout cables in a single order) as two of the ports on each of the cables appear to be faulty - they were sold as new by an Aussie Ebayer so am discussing with him at the moment.

This still does not explain the same issues on the 540Ts on the 1/3/x ports - i am waiting on some more 10G SFP to RJ45 transceivers to arrive to do further testing

Craig

 

[nextrack68]

Post up what you have in terms of the switch config

Have you set a default gateway on the switch - which should point to your PFSense

Presumably you have a trunk setup to the Pfsense box to carry both of the VLANs ?

Also show some screenshots of what you have in Pfsense

Essentially you should have a tagged port on the 6450 in both VLAN 20 and 30 and this should connect to PFsense which also has that port defined as Tagged with the two VLANs

Craig

Craig

SSH@brocade>show run
Current configuration:
!
ver 08.0.30uT313
!
stack unit 1
module 1 icx6450-48p-poe-port-management-module
module 2 icx6450-sfp-plus-4port-40g-module
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
!
!
!
aaa authentication web-server default local
aaa authentication login default local
hostname brocade
ip dhcp-client disable
ip dns server-address 192.168.1.1
ip route 0.0.0.0/0 192.168.1.1
ip route 192.168.1.0/24 192.168.30.0
ip route 192.168.30.0/24 192.168.1.1
!
no telnet server
username admin password .....
snmp-server community ..... ro
!
!
clock summer-time
clock timezone gmt GMT-06
!
!
ntp
disable serve
server 216.239.35.0
server 216.239.35.4
!
!
web-management https
web-management session-timeout 900
!
!
!
interface ethernet 1/1/6
inline power power-by-class 3
!
interface ethernet 1/1/7
inline power priority 1 power-by-class 3
!
interface ve 1
ip address 192.168.1.239 255.255.255.0
ip address 192.168.1.2 255.255.255.0 secondary
!
!
!
!
!
!
!
!
!
end


I really appreciate the help. For reference, I had a small 8 porch switch with the same pfSense box that I was able to run VLANs on just fine. The VLANs have not changed, I am just trying to adapt them to they new 6450p. I am losing my mind.

I do not know how to set the default gateway. pfSense is at 192.168.1.1

I also do not have a clue what the trunk is.

Finally, can the 6450p handle multiple VLANs coming through on one port? I need Port 6 to carry VLANs 20 & 30. Each is a different wireless SSID. I also need the normal (no vlan?) network to function fine too both in wifi and wired. The 192.168.100 through 200 addresses.

What would you like to see specifically in pfSense and I'll get screenshots. I am much more comfortable with it.

Thanks

 

[Craig Curtin]

SSH@brocade>show run
Current configuration:
!
ver 08.0.30uT313
!
stack unit 1
module 1 icx6450-48p-poe-port-management-module
module 2 icx6450-sfp-plus-4port-40g-module
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
!
!
!
aaa authentication web-server default local
aaa authentication login default local
hostname brocade
ip dhcp-client disable
ip dns server-address 192.168.1.1
ip route 0.0.0.0/0 192.168.1.1
ip route 192.168.1.0/24 192.168.30.0
ip route 192.168.30.0/24 192.168.1.1
!
no telnet server
username admin password .....
snmp-server community ..... ro
!
!
clock summer-time
clock timezone gmt GMT-06
!
!
ntp
disable serve
server 216.239.35.0
server 216.239.35.4
!
!
web-management https
web-management session-timeout 900
!
!
!
interface ethernet 1/1/6
inline power power-by-class 3
!
interface ethernet 1/1/7
inline power priority 1 power-by-class 3
!
interface ve 1
ip address 192.168.1.239 255.255.255.0
ip address 192.168.1.2 255.255.255.0 secondary
!
!
!
!
!
!
!
!
!
end


I really appreciate the help. For reference, I had a small 8 porch switch with the same pfSense box that I was able to run VLANs on just fine. The VLANs have not changed, I am just trying to adapt them to they new 6450p. I am losing my mind.

I do not know how to set the default gateway. pfSense is at 192.168.1.1

I also do not have a clue what the trunk is.

Finally, can the 6450p handle multiple VLANs coming through on one port? I need Port 6 to carry VLANs 20 & 30. Each is a different wireless SSID. I also need the normal (no vlan?) network to function fine too both in wifi and wired. The 192.168.100 through 200 addresses.

What would you like to see specifically in pfSense and I'll get screenshots. I am much more comfortable with it.

Thanks

OK so a couple of answers

A trunk is a "special" port that you setup to carry multiple VLANs - nothing extra - most of the time you will have a trunk with Tagged VLANs and possibly 1 untagged VLAN but for your case you just need two tagged VLANs.

I have not done Pfsense for a long time (i use OpnSense now) - but the basics are that you designate a port as a trunk and then you create sub interfaces on there - each of the sub interfaces is the VLAN that will go over that trunk

This looks pretty good

How to Set up a VLAN in pfSense

This tutorial will take a look at how to set up a VLAN in pfSense. VLANs (Virtual LANs) in pfSense are a way to easily separate traffic on your local network. In most traditional home

www.wundertech.net

On the 6450 end of things - you setup the Port to be a member of multiple VLANs - so in your case

Login to the switch
= username and password

go to enable mode
en

then
conf t

now to create a VLAN you just type

VLAN 20 name Wireless-20
tag eth 1/1/6

then

VLAN 30 name Wireless 30
tag eth 1/1/6


to set the default route

ip route 0.0.0.0/0 192.168.1.1


Presumably you will also have the Wireless APs attaching to ports on this switch - so they will need to be setup as tagged ports as well as per the above

If you want the default VLAN (being VLAN 1) to be available then you would need to setup the trunk ports for dual-mode and add them to the default VLAN in untagged mode (its not really recommended to use the default VLAN 1 for traffic

WHat i usually do is create another VLAN (call is 4000) and designate it as the Default VLAN - then you have VLAN 1 free to add untagged ports and dual mode to

Craig

 

[kevindd992002]

@fohdeesha so I'm following your guides in updating the device firmware, bootloader, and poe firmware of my ICX6450. I'm already at the poe firmware part and this is what I got:

ICX6450-48P Router#inline power install-firmware stack-unit 1 tftp 192.168.20.21                                                                                                                                                                                                                                              ICX64xx/icx64xx_poeplus_02.1.0.b004.fw
ICX6450-48P Router#Flash Memory Write (8192 bytes per dot) ..........
tftp download successful file name = poe-fw
Sending PoE Firmware to Unit 1.
Firmware version from File: 2.1.1
PoE Warning: Upgrading firmware in slot 1....DO NOT HOTSWAP OR POWER DOWN THE MO                                                                                                                                                                                                                                             DULE.
PoE Info: FW Download on slot 1...sending download command...
PoE Info: FW Download on slot 1...TPE response received.
PoE Info: FW Download on slot 1...sending erase command...
PoE Info: FW Download on slot 1...erase command...accepted.
PoE Info: FW Download on slot 1...erasing firmware memory...
PoE Info: FW Download on slot 1...erasing firmware memory...completed
PoE Info: FW Download on slot 1...sending program command...
PoE Info: FW Download on slot 1...sending program command...accepted.
PoE Info: FW Download on slot 1...programming firmware...takes around 12 minutes                                                                                                                                                                                                                                             ....
U1-MSG: PoE Info: Firmware Download on slot 1.....10 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....20 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....30 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....40 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....50 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....60 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....70 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....80 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....90 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....100 percent completed.
PoE Info: FW Download on slot 1...programming firmware...completed.
PoE Info: FW Download on slot 1...upgrading firmware...completed. Module will be                                                                                                                                                                                                                                              reset.
PoE Info: Resetting in slot 1....
PoE Info: Resetting module in slot 1....completed.
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev fault
PoE Info: Hard Resetting in slot 1....
PoE Info: Resetting module in slot 1....completed.
PoE Info: Programming Ruckus defaults. Step 1: Writing port defaults on module i                                                                                                                                                                                                                                             n slot 1....
PoE Info: Programming Ruckus Defaults: Step 2: Writing PM defaults on module in                                                                                                                                                                                                                                              slot 1.
PoE Info: Programming Ruckus defaults. Step 3: Writing user byte 0xf3 on module                                                                                                                                                                                                                                              in slot 1.
PoE Info: Programming Ruckus defaults. Step 4: Saving settings on module in slot                                                                                                                                                                                                                                              1.
PoE Info: Programming Ruckus defaults....completed.
PoE Info: PoE module 1 of Unit 1 initialization is done.

ICX6450-48P Router#write memory
ICX6450-48P Router#reload
Sent SIGTERM to all processesn'):
Sent SIGKILL to all processes
Requesting system reboot
Restarting system.

Questions:

1. Are those poe errors expected? It's worth noting that I don't see the same errors when booting (the poe modules are initialized just fine). So these two errors only ever showed during the poe firmware update.

Spoiler: Boot logs

Bootloader Version: 10.1.05T310 (Mar 19 2015 - 16:39:59)


Model ID: 1.0.0.1.1.0

Enter 'b' to stop at boot monitor: 0
bootdelay: ===
Booting image from Primary
## Booting image at 00007fc0 ...
Created: 2020-04-23 17:58:12 UTC
Data Size: 9870536 Bytes = 9.4 MB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum ... OK
OK

Starting kernel in BE mode ...
Uncompressing Image................................................................................................................................................................................ ................................................................................................................................................................................................... ........................... done, booting the kernel.
Config partition mounted.
Creating TUN device
Starting the FastIron.
FIPS DisabledORT NOT DISABLED
platform type 47
OS>Unable to set the kernel wall time
Starting Main Task .CPSS DxCh Version: cpss3.4p1 release
Pre Parsing Config Data ...

Parsing Config Data ...
Enable optical monitoring and set alarm/warn interval to default(3 minutes)

System initialization completed...console going online.
Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
UNIT 1: compiled on Apr 23 2020 at 10:57:06 labeled as ICX64R08030u
(9871112 bytes) from Primary ICX64R08030u.bin
SW: Version 08.0.30uT313
Boot-Monitor Image size = 786944, Version:10.1.05T310 (kxz10105)
HW: Stackable ICX6450-48-HPOE
==========================================================================
UNIT 1: SL 1: ICX6450-48P POE 48-port Management Module
Serial #: 2ax5o2jk68e
License: ICX6450_PREM_ROUTER_SOFT_PACKAGE (LID: H4CKTH3PLN8)
P-ENGINE 0: type DEF0, rev 01
P-ENGINE 1: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
512 MB DRAM
STACKID 1 system uptime is 28 second(s)
The system started at 08:00:38 GMT+08 Thu Jan 01 1970

The system : started=warm start reloaded=by "reload"

ICX6450Switch>
Stack unit 1 PS 1, Internal Power supply detected and up.

Stack unit 1 PS 1, Internal Power supply detected and up.
PoE: Stack unit 1 PS 1, Internal Power supply with 740000 mwatts capacity is up
PoE Info: Adding new 54V capacity of 740000 mW, total capacity is 740000, total free capacity is 740000
PoE Info: PoE module 1 of Unit 1 on ports 1/1/1 to 1/1/48 detected. Initializing....
PoE Event Trace Log Buffer for 2000 log entries allocated
PoE Event Trace Logging enabled...
PoE Info: PoE module 1 of Unit 1 initialization is done.

2. In your guide you mentioned

"#after a few seconds, hit enter to return to cli".

I did this after I saw this message "PoE Info: PoE module 1 of Unit 1 initialization is done." Is that fine?

3. In your guide, you mentioned:

#you'll probably get a message that it hasn't finished. it can take up to 10 minutes
#run "show log" occasionally to monitor the update progress
#try the "reload" command again once it's reached 100%

I didn't see this message. Is this because I hit enter when the poe firmware update process is already done?

4. What's the difference between reload and reset? I see that they both reboot the switch anyway.

 

[baskethammer]

I recently purchased a 6450 and followed the guide. Everything seems to work and i'm running on v. 08.0.30uT313.

For the life of me I cannot figure out how to assign an ip address to a given port:

SSH@chonk(config)#int eth 1/1/48          
SSH@chonk(config-if-e1000-1/1/48)#ip address 10.10.10.1/24
Invalid input -> address 10.10.10.1/24

The attempt is in line with the documentation for that version:

Commscope Technical Content Portal

docs.commscope.com

And the various 6450 configuration videos out there. I was trying to approximate @kapone's approach in his excellent guide.

Thanks for any help demystifying this.

3. Your home/whatever VLAN where the (some of the) devices are - Add whatever untagged ports you need on the switch but also add the transit port as tagged. This will ensure that any unknown traffic from this VLAN will be tagged by the time it hits pfSense over the transit pipe.

In my case, on the switch side, I added a few untagged ports to this, added the tagged transit port and gave it an IP of 10.10.10.1/24. That IP address is important...

 

[bitbckt]

The docs aren’t using CIDR masks in your example. Have you tried 255.255.255.0 instead?