Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[John T Davis]

Thanks!

It sounds like the easiest thing to do is just to assign a static IP to the management port that's in VLAN 10's range, and plug it into a port on my upstream switch that's untagged on the management VLAN. That way the ICX management port doesn't have to be VLAN aware. Right?

I've done a bit more research, and it looks like on the latest version of FastIron, disabling SSH is as simple as ip ssh disable.

 

[SeRiusMe]

1. Start reading at the beginning of this thread.
2. Continue reading till the end.
3. Go back and do it at least twice more.

If you still haven't found your answers, I'll be happy to help you.

Now, I just ended my 2nd round and reached this page again.
I've already done almost all the configuration, but I still struggle on the opnsense side. I've seen that some people talk about FW rules, that I assume are to place on the TRANSIT vlan. We talked about this last time.
But other mentions and shows NAT outbound rules. And I don't know what should I use.

The question is because I fear doing something bad on the WAN side and expose my network. And on the other side, I tried gaining access to the new Management VLAN being on the old LAN by configuring an (MGMT segment) allow (OLD LAN network) on the TRANSIT VLAN and failed miserably.
Yes, I still have old network coexisting with the new.

 

[John T Davis]

Thanks for the help. I got the management interface up and running with a simple static IP assignment:

interface management 1
ip address 12.34.56.78/24

[after making sure I could ping it:]
write mem

Looking at the Advanced config, I saw this bit:

If you want your switch to be able to contact NTP servers for time synchronization, remote SNMP servers, etc, we need to give the switch a default route and a DNS server. Replace the IP with the IP of your gateway/router/etc. Assuming you are still at the configure terminal level:

ip dns server-address 192.168.1.1
ip route 0.0.0.0/0 192.168.1.1


During initial setup, I used VLAN 1's DNS and gateway for this. Now that I have the management port working with a static IP, can I use that subnet's DNS and gateway, instead? That is, is the switch able to use the management port to contact NTP and SNMP servers, or does it always try to use ve 1?

(Yes, I can just try this, but I thought it might be useful to ask here for those who find this thread later, or in case even though it "works" for me there's some potential pitfall I'm not aware of.)

EDIT: Okay, that seems to work. Caveat: VLAN 10 (10.10.10.x) is tagged on my uplink SFP+ port, but not tagged on any of the 1 Gbps switch ports or (obviously) the management port.

janice-iii(config)#ip dns server-address 10.10.10.1
janice-iii(config)#ip route 0.0.0.0/0 10.10.10.1
janice-iii(config)#exit [can't ping in config terminal]

janice-iii#ping google.com

Type Control-c to abort
Sending DNS Query to 10.10.100.1
Sending 1, 16-byte ICMP Echo to 142.250.115.100, timeout 5000 msec, TTL 64
Type Control-c to abort
Reply from 142.250.115.100 : bytes=16 time=8ms TTL=105
Success rate is 100 percent (1/1), round-trip min/avg/max=8/8/8 ms.

N.B.: Make sure to write mem after testing this.

EDIT 2: Okay, that doesn't actually work like I thought it would. The original DNS and route for VLAN 1 are still there. Take two: I left the DNS and route for VLAN 1 in place, and removed the IP from ve 1.

That was probably not what we're supposed to do, as suddenly even though I still had DNS resolution, I could no longer ping out because ve 1 no longer had an IP to receive replies on.

That doesn't bother me hugely … I can always get in and add an IP back to VLAN 1 if I need to troubleshoot with ping later. I need to figure out if removing the IP from ve 1 breaks NTP.

EDIT 2(a): Removing the IP from ve 1 also breaks NTP sync. Annoying, but not a killer problem … I think?

It'd be better if I could just tell it not to listen for HTTP(S) or SSH on ve 1, as mostly I just don't want the admin interfaces exposed there. Is there a way to do that?

Or is there some better way to config this that I haven't figured out yet?

 

[chlastakov]

You probably overthinking this. Remove all IPs from ve's. Remove all ve's. Keep only IP on management port and set router for gateway IP on your management network.

I set this this way and everything works great.

 

[kpfleming]

You probably overthinking this. Remove all IPs from ve's. Remove all ve's. Keep only IP on management port and set router for gateway IP on your management network.

I set this this way and everything works great.

Yes, that's what you want if your goal is to operate the switch in layer 2 mode, not layer 3 mode. Keep it simple.

 

[jcjefamilyfun]

Good morning everyone!
a little about me. I am a new member.
My name is john and I am not really new to networking, but i am not good at networking. I am here at the recommendation of @fohdeesha.
He told me i could lean from you all.
I have a icx 6450 c12 PD actually I have three of them.
A pfsense firewall(not hooked up because i am still in the process of moving).it is installed on a Dell 7050 micro computer.
A Creality Cr-m4 3d printer this has to be on the 10.10.10.x network for some reason
a windows box and a ubuntu box. 192.168.1.x network

I would like to hear what you all think of my collection of stuff, and what you think i should do with it.
i know that is a loaded question, so I won't be sticking it somewhere and i wont be giving it away or throwing it away.

 

[John T Davis]

Thank you both. I'm actually glad to hear I was overcomplicating it--it sure felt like that's what I was doing, but I'm a bit too new with this gear to have been sure. I was followiong the tutorial in the Fohdeesha docs, but I suspect after reading your messages that that its default config is probably intended to be a core switch, if it's using layer 3.

I'm … pretty sure … I only need level 2 functionality? This is an edge switch. Pretty much, I just wanted extra 1 Gbps ports with PoE behind my desk. My upstream core switch is only an L2 switch that can't actually do L3 stuff, and my firewall is an OPNSense box. My VLANs and firewall are configured to almost entirely avoid inter-VLAN routing.

A few questions for setting it up in L2 mode:

1. I've got an upstream dns server and default route set in my config. Do I still need those?
2. "set router for gateway IP on your management network" >> I'm not exactly sure what this means. Is that the router-interface?

EDIT: I did get this working. Thanks for the pointers to get going in the right direction. After work, I'll come back and document how I fixed everything, in case someone finds it useful.

1. I removed the IP from the VE and then removed the VE itself.
2. I assigned an IP on the correct subnet to the management port.
3. Ping commands and the NTP client failed at this point.
4. I set the DNS server to the upstream gateway/firewall address on the correct subnet.
5. I set the correct default route for the subnet.

After that, everything worked: I can ping out from the switch, and the NTP client works again.

 

[John T Davis]

So … after consulting the documentation, I have a question about jumbo frames.

I have a storage VLAN that's set to use MTU 9000 jumbo frames (my Mac client has an Aquantia NIC on board and absolutely falls on its face if I try to get it to do 10 Gbps without jumbo frames, so I standardized on that on the storage VLAN).

So, I'd like to enable jumbo frames only on the storage VLAN.

After looking through the Ruckus documentation PDFs, I'm still not 100 percent sure how to do this correctly. Here's what I've figured out so far. I'm not sure if it's all correct.

1. I'd use the jumbo command to enable the use of jumbo frames switch-wide.

…Sadly, that's the limit of what I completely understand. I'm still not sure how to (1) set MTU 1500 on all non-storage VLANs and the management port; and (2) set MTU 9000 on the storage VLAN.

I'd rather not just guess at this point. I'd really appreciate any help. This is the last thing I want to configure on the switch.

 

[kpfleming]

So … after consulting the documentation, I have a question about jumbo frames.

I have a storage VLAN that's set to use MTU 9000 jumbo frames (my Mac client has an Aquantia NIC on board and absolutely falls on its face if I try to get it to do 10 Gbps without jumbo frames, so I standardized on that on the storage VLAN).

So, I'd like to enable jumbo frames only on the storage VLAN.

After looking through the Ruckus documentation PDFs, I'm still not 100 percent sure how to do this correctly. Here's what I've figured out so far. I'm not sure if it's all correct.

1. I'd use the jumbo command to enable the use of jumbo frames switch-wide.

…Sadly, that's the limit of what I completely understand. I'm still not sure how to (1) set MTU 1500 on all non-storage VLANs and the management port; and (2) set MTU 9000 on the storage VLAN.

I'd rather not just guess at this point. I'd really appreciate any help. This is the last thing I want to configure on the switch.

These switches do not offer per-VLAN control over the frame size. It's a full-switch configuration only.

However, the setting on the switch only controls what the switch will permit, it does not cause the connected devices to use larger frames. If the devices connected to the non-storage VLANs are not configured to use MTUs larger than 1500 bytes, they won't.

 

[John T Davis]

This is exactly what I needed. The documentation I've been looking at implied that I needed to set the MTU on each L3 interface (VLAN, management port, etc.), but the switch wouldn't let do that. One thing I read implied that after enabling jumbo frames, the management port would try to use MTU 9000 unless I explicitly set it to MTU 1500, since it's an L3 interface.

I've enabled jumbo frames, so it sounds like I'm good.

(I might never actually even use my storage VLAN on this switch, but I just wanted to make sure I could.)

I'm excited. For the moment, I'm actually done configuring this thing. Thanks!

 

[titoum]

hi there,

i just bought on ebay a Ruckus ICX6450-24 24-Port Gigabit Ethernet Switch with 4 x 10G SFP+ Ports and the description was stating that it was supporting POE.

i had a look at datasheet and in my opinion it is only the P version that support it or am i wrong?
Does the not P one can do it with the external power supply or was the description misleading?

i got it for 156eur but if description is incorrect then i will bargain with the seller.

thank you in advance for your help

 

[itronin]

i just bought on ebay a Ruckus ICX6450-24 24-Port Gigabit Ethernet Switch with 4 x 10G SFP+ Ports and the description was stating that it was supporting POE.

i had a look at datasheet and in my opinion it is only the P version that support it or am i wrong?

specs

ICX6450-24p has POE.
ICX6450-24 does not have POE

 

[titoum]

ICX6450-24p has POE.
ICX6450-24 does not have POE

thx! unfortunately, i understood well then... so this is when the fun begins :-/

 

[aarcane]

So I know a lot of people are happy just mounting these by the ears, and that's excellent if you have it set on something stationary, or just hanging freely on its own, but if you've got sliding servers underneath it, you don't want any sag on the device. Your options are generic "Shelf" style rails, which work fine if you've got 2u available for shelf stackable things. The official rail kit is nearly impossible to find on ebay, and most of these come with 0 or 2 ears for hanging. So what is one to do if you need to mount something underneath it?

Go shopping for generic rail kits that have the right holes! Well, it just so happens that I bought some rails for an older fibre channel switch that I wasn't using, and decided to try them on this switch. They don't line up with *all* the holes, but they do have 2 that fit *PERFECTLY*, and they're generally widely available for cheap on E-bay, and have been for years. Obviously, you won't be pulling your switch in and out of the rack every week to tinker, but rails are still incredibly useful, so here's the one I had on my icx6610-48p for the last few years until I just decommissioned it a few months back.

Citrix P/N 8530007 Universal Rail, 4 Post Rack 28 - 38 Inches

 

[chlastakov]

So I know a lot of people are happy just mounting these by the ears, and that's excellent if you have it set on something stationary, or just hanging freely on its own, but if you've got sliding servers underneath it, you don't want any sag on the device. Your options are generic "Shelf" style rails, which work fine if you've got 2u available for shelf stackable things. The official rail kit is nearly impossible to find on ebay, and most of these come with 0 or 2 ears for hanging. So what is one to do if you need to mount something underneath it?

Go shopping for generic rail kits that have the right holes! Well, it just so happens that I bought some rails for an older fibre channel switch that I wasn't using, and decided to try them on this switch. They don't line up with *all* the holes, but they do have 2 that fit *PERFECTLY*, and they're generally widely available for cheap on E-bay, and have been for years. Obviously, you won't be pulling your switch in and out of the rack every week to tinker, but rails are still incredibly useful, so here's the one I had on my icx6610-48p for the last few years until I just decommissioned it a few months back.

Citrix P/N 8530007 Universal Rail, 4 Post Rack 28 - 38 Inches

I went with this: https://www.amazon.com/dp/B0060RUVBA?ref=ppx_yo2ov_dt_b_fed_asin_title

 

[aarcane]

I went with this: https://www.amazon.com/dp/B0060RUVBA?ref=ppx_yo2ov_dt_b_fed_asin_title

Yup. Classic shelf style mount. Turns any 1u device into a 2u device with some storage space on top! Great if you need somewhere to put your modem!

 

[titoum]

a big thanks to @fohdeesha for all the work/documentation. all went as a breeze

 

[chlastakov]

I will join with thanks. You did amazing work @fohdeesha and I thank you for that. I am very happy with my new Brocade switches

 

[i386]

@fohdeesha made me spent money on icx switches and ruckus aps

 

[titoum]

mmmm i have a tricky question...

is it safe this coil whine when fan is plugged in? i have it on both socket...couldnt hear before because of fan noise