Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[JacobRhea]

10.0.0.3 is the broadcast address for your 10.0.0.2/255.255.255.252 subnet, which is in the routing table as reachable via 1/1/47. You've also set 1/1/47 as 'route-only' so it is not part of VLAN 1.

You will need to assign an address outside of that subnet to the R710, but then you'll also need a VE for VLAN 1 which makes that other subnet known to the ICX, or some other device outside of the ICX will have to do it.

I knew the reason I couldn't assign 10.0.0.3 was because it's part of that broadcast address. Is there anyway I can change that so I can use 10.0.0.1 for the router, 10.0.0.2 for the switch and 10.0.0.3 for the access point?

So I need to assign an adress to the R710 that is the same subnet as the VE I assign to VLAN1?

 

[cesmith9999]

You will need to make the subnet larger

Chris

 

[kpfleming]

Just making the subnet larger won't be sufficient, as right now that subnet is only reachable via 1/1/47. You have setup 1/1/47 as a layer 3 interface and that subnet cannot also be present on any other interfaces.

Here's what I would do (and you'll need to do this using the serial console, or an SSH connection to the ICX that is *not* over the 10.0.0.2 address):

1. Remove 'route-only' and 'ip address' from 1/1/47. This will revert 1/1/47 back to being a layer 2 interface in VLAN 1.
2. Add VE 1, with 'ip address 10.0.0.2 255.255.255.248' (this makes the subnet larger, so that you can use host addresses 10.0.0.1 through 10.0.0.6).
3. Configure the R710 with address 10.0.0.3 with subnet mask 255.255.255.248.
4. Change the subnet mask on your modem to 255.255.255.248.

With that done the modem, the ICX, and the R710 will all have addresses in the 10.0.0.0/255.255.255.248 subnet, and the ICX will have *two* layer 2 ports in that subnet (1/1/47 and 1/1/48) so that all three devices can talk to each other on that VLAN/subnet.

In the future you could add three more devices on VLAN 1 with addresses from the range 10.0.0.4 - 10.0.0.6. You could also simplify your life by just using 255.255.255.0 as the subnet mask for the VLAN 1 subnet like you are using for all of the other subnets, then you'd be free to as many 10.0.0.x addresses as you want (up to 254 of course).

 

[JacobRhea]

kpfleming

Thank you so much for this post! I'll work on setting it all up later tonight.

 

[vinegar]

Word of warning, there are still noisy 6450 models out there.

Just got my ICX6450-48 and hit a bad luck with fans, much like @u238 and @tony& 4 years ago. Fans are running full speed, not coming down, despite chassis info showing speed 1 (low). Even if I replace fans, it will still be running at full speed, still being very audible

ICX6450-48-Router>show chassis
The stack unit 1 chassis info:

Power supply 1 (NA - AC - Regular) present, status ok
Power supply 2 not present

Fan ok, speed (auto): [[1]]<->2

Fan speed switching temperature thresholds:
        1 -> 2 @ 69 deg-C
        1 <- 2 @ 64 deg-C

Sensor B Temperature Readings:
        Current temperature : 26.5 deg-C
Sensor A Temperature Readings:
        Current temperature : 27.5 deg-C
        Warning level.......: 66.0 deg-C
        Shutdown level......: 76.0 deg-C
Boot Prom MAC : cc4e.2453.5b80
Management MAC: cc4e.2453.5b80

Version information shown after boot:

  Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
    UNIT 1: compiled on Apr 23 2020 at 10:57:06 labeled as ICX64R08030u
                (9871112 bytes) from Primary ICX64R08030u.bin
        SW: Version 08.0.30uT313
  Boot-Monitor Image size = 786944, Version:10.1.05T310 (kxz10105)
  HW: Stackable ICX6450-48
==========================================================================
UNIT 1: SL 1: ICX6450-48 48-port Management Module
         Serial  #: BZU0451J01N
         License: ICX6450_BASE_ROUTER_SOFT_PACKAGE   (LID: dbwFJKGlFGp)
         P-ENGINE  0: type DEF0, rev 01
         P-ENGINE  1: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
  800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1  system uptime is 27 second(s)

 

[Glock24]

Would these work on 7150-c12p or 6450-24 switches? Would they work on Connect-X3 cards?

FS SFP-10GSR-85 CO SFP+ 850nm 300m SR SFP Transceiver Module (Lot of 4) | eBay

Find many great new & used options and get the best deals for FS SFP-10GSR-85 CO SFP+ 850nm 300m SR SFP Transceiver Module (Lot of 4) at the best online prices at eBay! Free shipping for many products!

www.ebay.com

 

[fohdeesha]

Would these work on 7150-c12p or 6450-24 switches? Would they work on Connect-X3 cards?

FS SFP-10GSR-85 CO SFP+ 850nm 300m SR SFP Transceiver Module (Lot of 4) | eBay

Find many great new & used options and get the best deals for FS SFP-10GSR-85 CO SFP+ 850nm 300m SR SFP Transceiver Module (Lot of 4) at the best online prices at eBay! Free shipping for many products!

www.ebay.com

if they were actually what he listed in the title yes, however his pictures are of XFP modules which are totally different and will not fit or work in an SFP slot

 

[Craig Curtin]

Stack Issues with new unit - 6610-48-POE

Guys,

This is my 2nd setup of a 6610 - the previous one went smoothly (was not POE)

This one i am having problems - the unit believes it is part of a stack (or at least retains the old config in terms of Unit ID - being 3)



I have followed the section in the doc if my switch thinks it is still a stack member and entered the commands



Wondering if there are any thoughts on how i reset the unit ID ?

I have done a fair bit of googling and tried various things such as setting the unit ID and the Stack suggested-id but nothing seems to be working

Any pointers ?

Craig

 

[Glock24]

if they were actually what he listed in the title yes, however his pictures are of XFP modules which are totally different and will not fit or work in an SFP slot

Thanks for the answer. I didn't notice the pictures showed a different item from the description and I didn't know XFP even existed!

 

[fohdeesha]

Stack Issues with new unit - 6610-48-POE

Guys,

This is my 2nd setup of a 6610 - the previous one went smoothly (was not POE)

This one i am having problems - the unit believes it is part of a stack (or at least retains the old config in terms of Unit ID - being 3)

View attachment 25042

I have followed the section in the doc if my switch thinks it is still a stack member and entered the commands

View attachment 25043

Wondering if there are any thoughts on how i reset the unit ID ?

I have done a fair bit of googling and tried various things such as setting the unit ID and the Stack suggested-id but nothing seems to be working

Any pointers ?

Craig

connected to the switch over serial, once it's fully booted, press ctrl+y then let go, then press m, then enter. you should drop to an os console. then run the following commands

del stacking.boot
del startup-config
del startup-config.backup
del startup-config.legacy
reset

it'll reboot and be clean

 

[Craig Curtin]

connected to the switch over serial, once it's fully booted, press ctrl+y then let go, then press m, then enter. you should drop to an os console. then run the following commands

del stacking.boot
del startup-config
del startup-config.backup
del startup-config.legacy
reset

it'll reboot and be clean

Thank you very much for the quick response - onto it now - and about to try

Might be an idea to update your master doc with this gem ?

Craig

 

[Craig Curtin]

Thank you very much for the quick response - onto it now - and about to try

Might be an idea to update your master doc with this gem ?

Craig

Yep that got it thanks @fohdeesha - you are a legend Sir !

Craig

 

[JohnyDanger]

I have an icx6450-24p, and I have rack mounting ears. However, after drilling out all of the stripped old rack ear screws to swap the fans, I am now without rack ear screws. Anyone know what size they are?

 

[tls]

ICX6450 to ICX7150 - multi-gig SFP+

Are the Mikrotik S+RJ10 multi-gig SFP+ modules expected to work in an ICX6450? In a 7150 with up to date FastIron? At speeds other than 1G and 10G?

I just bought a pair and tried to deploy them between a 6450-24P and 7150-C12P on a roughly 100' cat6 run through 2 patch panels across my house - I'd expect this run to be 2.5G or maybe 5G capable but certainly not 10G - and they won't link up.

Interestingly, if I pull the multi-gig module from the 7150 end and replace with a 1G module, then the link does eventually come up, even though I do not have speed-duplex set to 1000. But with a multi-gig module at both ends, nada.

On the 7150, I see something very odd when I put one of the modules into a slot and show media ethernet:

edge-office>show media ethernet 1/3/2
Port   1/3/2: Type  : 1GE M-SX(SFP)
         Vendor: OTHERSik             Version: 2.16
         Part# : S+RJ10               Serial#: HCK01PA361Q

"OTHERSik"?

I did notice I had to insert the module very firmly or the slot just showed as EMPTY. But it's definitely seated properly now, and the other identical module I have also shows as an OTHERSik.

 

[Craig Curtin]

Mellanox COnnect CX3 to 6610

Guys, my first foray into 10Gbe (and higher) - i have followed along with the tutorial and got a couple of 6610 up and running fine. I want to now fitout a couple of machines with 10Gbe connections and am hoping to use the breakout ports on the rear (1/2/2-1/2/5 and 1/2/7 - 1/2/10) for 10GBe connections to a couple of servers in y home lab.

I am interested in the Mellanox CX3 (OEM) cards avaiable on ebay in australia as they are relatively cheap - i am just wondering though - is the physical slot just a SFP+ slot on these cards - so i can get one of the QSFP+ to SFP+ 4 way breakout cables - to allow me to connect 4 of the servers (at 10Gb) into one of the breakout cables - or is it a different connector on the Mellanox ?

Craig

 

[mach3.2]

i can get one of the QSFP+ to SFP+ 4 way breakout cables - to allow me to connect 4 of the servers (at 10Gb) into one of the breakout cables - or is it a different connector on the Mellanox ?

This works, but you'll have to use the 40GbE ports that can breakout into 4* 10GbE connections.

On the ConnectX 3 end, get the SFP+ version(CX311/CX312) instead of the QSFP+ 40GbE version(CX353/CX354).

If you have the 40GbE ConnectX 3 cards, you'll need QSA adapters and those cost around 20 bucks a pop.

 

[clifford64]

Maybe I am just not understanding something, but what is the proper way to enable multicast routing on a 6450? It is my understanding that if I have the below two commands, this enables multicast routing.
ip multicast passive
ip pimsm-snooping

It is my understanding that it will perform listing for multicast requests and keep a cache of which multicast groups there are on different interfaces. So if a multicast is sent to a specific group, it will be forwarded to each interface within that group even if that interface is on a separate VLAN (as long as you have intervlan routing setup).

So then in theory, if vlan 5 and vlan 17 both have a common group, a multicast packet sent from vlan 5 to that group address would arrive on the devices in vlan 17 that are in that group right?

I am essentially trying to get UPNP requests to hit my upstream router from an internal VLAN.

 

[Craig Curtin]

This works, but you'll have to use the 40GbE ports that can breakout into 4* 10GbE connections.

On the ConnectX 3 end, get the SFP+ version(CX311/CX312) instead of the QSFP+ 40GbE version(CX353/CX354).

If you have the 40GbE ConnectX 3 cards, you'll need QSA adapters and those cost around 20 bucks a pop.

Thanks very much for the detailed response - i am now officially on the hunt !

Craig

 

[Rossatron]

So, I have the icx 6610, and decided to try the 40g ports (not the breakout, they are in use but not relevant to my question) so I bought an adaptor from startech that has a 40gb qsfp port. I’ve removed 1/2/6 from stack (stack was disabled anyways) added it to my home vlan. Plugged the two together with arista qsfp modules and a fancy fibre cable. Got a link up at 40gbs. But no network traffic. Couldn’t get a dhcp IP address. Check the dhcp server logs and it was being asked for an ip and it was sending it but it never gets to the device connected to 1/2/6. I manually gave the device on 1/2/6 an ip and I could then ping any device on my network. But that’s it, literally all I can do is ping. I can’t access scp, ssh, html. Literally nothing appears to get through. It’s on the same vlan as all other devices so I’m at a loss. It’s definitely not showing as being a stacking port. But I can’t for the life of me fathom where to start troubleshooting. I’ve tried brocade qsfp transceivers, a second fibre cable, a different nic all with the same results. I even tried 1/2/1 and had exactly the same results. Only time I’ve ever been able to ping but no network access has been due to vlan issues but I’ve triple checked it’s in the vlan. Have tried tagging the port instead of untagged and I can't even ping it then. I’ve seen a few others have had a similar issue from reading this thread but no actual solutions have been posted, and yes I did try rebooting everything, crazily enough rebooting the switch seems to fix a few issues! So yeah, anyone with any ideas?

After further diagnostics, using tagged lan ensures the client is on the arp table for the router. doing a "ping google.com" obtains an IP address from the dns but it can't actually ping the ip its given.

Conversely removing the tag and adding it as untagged, I am able too ping local computers, and can obtain ip addresses from dns server. The client still shows on arp table though albeit with the wrong hostname, (Same hostname as the first windows based client I tried the network card in) But the ping actually goes out to the service (i.e. google, Facebook)

Traceroute doesn't work in either case.

So from all of the above from what I can tell I may be looking at a faulty NIC as vlan works fine with the two breakout ports.

 

[pypypy]

I have a 6450-48p with a dead POE module. Decided to pull the module completely because I had issues with ports randomly going down and back up again... After doing so, the fan speed is stuck at 100%. How do I remove the POE firmware so the switch thinks its a non-poe?