Another thing to consider is that the NTP and DHCP servers on the ICX are very limited to compared to opnSense or other highly-capable router/firewall systems. They may be sufficient for what you need in this situation, but you may also find that their limitations interfere and you'll end up with the more complex configuration where the 'isolated VLAN' is extended across the network to your existing NTP and DHCP servers.
Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Thank you for this hint.
The limitation of the dhcp server I already discovered in the past. But for this purpose it is more than fine.
For ntp, my home lab is too simple to hit limitations. In my home network I use four physical machines to provide ntp, two of them are icx switches and one is the opnSense firewall. Beside internet based source they use each other as a source. So, in case of reboot and failing internet connection I can get a fast re-sync. I do not know if this is best practice. I never discovered issues and I do not need the precision of stratum 0.
Fair enough. You know your situation and what you expect from it.
I understand how adding the CCTV switch is adding an extra hop to the data to get to/from the router and an additional failure point, but I don't think it is a problem. With what is likely a relatively small number of devices on the CCTV VLAN (which are going to stay pretty consistent over time), I would suggest assigning static address for them anyway so any loss of DHCP service (regardless of the source - switch, router, Windows DHCP service, etc) can't result in the NVR system to failing to connect and therefore record the cameras. (There might be other failures that could prevent recordings, but the loss of DHCP won't be a problem). You'll still want DHCP on the VLAN, but it would be rarely used. NTP isn't so critical that loosing sync - even for days/weeks - wouldn't cause an issue either.
Last edited:
Yes, static IP is an alternative and the number of devices is small. I can handle.
For servers I always use static IP. Otherwise I like the comfort of central management by dhcp. With dhcp it also is easier to change IP ranges (it is a home lab) without getting locked out of headless devices. But you are right. For robustness, static IP would be the best.
It is't critical for the operation but I want it to get the right time in the recordings.
Here, updating the time manually would be a pain in the ass.
For info that's exactly what I wanted to do as well.
I bought some "foreign" cameras which I do not trust, but which seems to be of very good quality.
I put them on a VLAN, with no access to internet, no access to other VLAN and no access to other devices on the same VLAN, but they can be accessed from other VLAN (where the "safe" NVR is located).
I also needed the switch to have a NTP server, as the cameras needed to be in sync.
The DHCP server on the switch is working (tested with my computer) but somehow that doesn't work with the cameras (I don't know if this is the cameras or the switch DHCP server though).
So far everything is working perfectly.
It solved a lot of headaches, especially when the cameras are delivered configured with a fixed ip address which does not match what I am currently using in my network.
I just had to create a VLAN with this particular IP address range, and I could access the camera (of course without the cameras accessing anything else).
I could see in the logs, that the cameras are often trying to send queries on internet, so I feel much safer.
I bought some "foreign" cameras which I do not trust, but which seems to be of very good quality.
I put them on a VLAN, with no access to internet, no access to other VLAN and no access to other devices on the same VLAN, but they can be accessed from other VLAN (where the "safe" NVR is located).
I also needed the switch to have a NTP server, as the cameras needed to be in sync.
The DHCP server on the switch is working (tested with my computer) but somehow that doesn't work with the cameras (I don't know if this is the cameras or the switch DHCP server though).
So far everything is working perfectly.
It solved a lot of headaches, especially when the cameras are delivered configured with a fixed ip address which does not match what I am currently using in my network.
I just had to create a VLAN with this particular IP address range, and I could access the camera (of course without the cameras accessing anything else).
I could see in the logs, that the cameras are often trying to send queries on internet, so I feel much safer.
I saw in the logs:
Is that the problem you are talking about with the Brocade ICX NTP server and why we should not rely on it ?
Code:
Dec 22 12:09:32:I:NTP: System clock is synchronized to 134.130.4.17.
Dec 22 12:08:59:I:NTP: The system clock is not synchronized to any time source.With
show ntp associations and show ntp status you can get more information about your ntp server on icx.
Code:
SSH@switch02>show ntp associations
address Domain name Reference Clock st when poll Reach delay offset disp
+~ 144.76.59.37 0.de.pool.ntp.org 195.145.119.188 2 40 64 377 16.223 5.0091 4.882
+~ 45.145.40.190 1.de.pool.ntp.org 42.218.218.254 3 18 64 377 15.222 3.0090 1.662
+~ 93.241.86.156 2.de.pool.ntp.org 195.145.119.188 2 45 64 377 13.002 8.1191 5.009
*~ 212.18.3.19 3.de.pool.ntp.org 195.145.119.188 2 26 64 377 13.662 3.7728 1.770
+~ 192.168.2.1 None 237.17.204.95 2 32 64 177 2.441 4.2200 4.220
+~ 192.168.2.2 None 192.168.2.1 3 19 64 377 2.550 23.1142 4.225
~ 192.168.2.10 None STEP 16 - 1024 0 0.00 0.000 15937.
* synced, # selected, + candidate, - outlayer, x falseticker, ~ configured, **More characters in domain name
SSH@switch02>
Code:
SSH@switch02>show ntp status
Clock is synchronized, stratum 3, reference clock is 212.18.3.19
precision is 2**-16
reference time is 3975429296.423812614 (22:54:56.423812614 GMT+01 Mon Dec 22 2025)
clock offset is 6.8886 msec, root delay is 15.0074 msec
root dispersion is 10.8804 msec, peer dispersion is 0.1198 msec
system poll interval is 64, last clock update was 99 sec ago
NTP server mode is enabled, NTP client mode is enabled
NTP master mode is enabled, NTP master stratum is 5
NTP is not in panic mode
SSH@switch02>I know but I haven't seen any big issues with NTP with those commands.
Sometimes (as seen in the log), I got this message "The system clock is not synchronized to any time source", but soon after it is synced again.
I was curious about the message:
For the DHCP server, I know that the server is not "authoritative".
Sometimes (as seen in the log), I got this message "The system clock is not synchronized to any time source", but soon after it is synced again.
I was curious about the message:
So I wanted to understand what is the limitation of the NTP server on the ICX.
For the DHCP server, I know that the server is not "authoritative".
The typical way to have isolated l3 networks on one router would be VRFs, but dunno if your model supports them, the 8200 is a huge range
Thank you. I will have a look in the topic VRF. So far, I am not familiar with.
It is an 8200-24P.
vlan's are used to separate layer2 traffic, which can be linked together by layer3 routing table. VRF's are used to seperate layer3 traffic by creating multiple independent routing tables.
I have been using Aruba S2500s in my house for quite some time, like at least 1 in every room, and, well kind of a lot of them in a few rooms, mostly because the number of 10G SFPs you net out of one of these is 2 if you ring stack them. I want to move LOTS more over to 10G so looking to replace them all with Brocades ICXs. I currently have 19 of the S2500 and I am using most of the 10G ports, with most nowhere near where I actually need them. This all seemed generous when I didn't have any 10G ports, now I am running longer and longer cables to connect things and I am at the point where there just isn't enough ports to do what I want to do no matter what I do.
So... Plan is to replace the switches in my office and lab with stacked 6610s, use a pair of 6650s for a dedicated storage and Proxmox replication network, and all the rest of the house with a 7250 stack. My main question is about the QSFP+ ports. I see that both the 6610s and 6650s have a mix of 40GB "only" ports and 4x10GB "breakout only" ports. I am assuming these need to be treated differently? Particularly in how they are treated as part of Port Channels, where the Brocade rules seem to indicate that you can't mix port speed within an LACP LAGG. I'm guessing this extends to interconnecting switches to each other, especially on the 6650s which don't stack.
Thinking this is probably the case I've evolved my thoughts on this to just doing a fully trunked stack on the 6610s, use the 4x10G QSFPs on the 6650s to create a port channel to some of the 6610 front 10G ports and use all the 40G native ports to create port channels between the 2 6650s. It's not the most elegant solution but seems like the best way to play within the rules and get big bandwidth between the 6610 stack and the 6650 not really a stack. The 7250s have a lot lower expectations, so I'll probably just reuse the Aruba 10G stack cabling.There is actually only 1 thing anywhere local to any of them that will even need 10G at all.
Anyway, just trying to confirm that my assumptions are correct, that the 40G ports are just that and that the breakout ports are really just logically 4 10Gs using a common connection and not really a 40Gb port. Is that correct? So I can connect 40s to 40s, and 10s, whether SFP+s or QSFP+s, together but never 40G natives to the 4x10G 40s, right?
So... Plan is to replace the switches in my office and lab with stacked 6610s, use a pair of 6650s for a dedicated storage and Proxmox replication network, and all the rest of the house with a 7250 stack. My main question is about the QSFP+ ports. I see that both the 6610s and 6650s have a mix of 40GB "only" ports and 4x10GB "breakout only" ports. I am assuming these need to be treated differently? Particularly in how they are treated as part of Port Channels, where the Brocade rules seem to indicate that you can't mix port speed within an LACP LAGG. I'm guessing this extends to interconnecting switches to each other, especially on the 6650s which don't stack.
Thinking this is probably the case I've evolved my thoughts on this to just doing a fully trunked stack on the 6610s, use the 4x10G QSFPs on the 6650s to create a port channel to some of the 6610 front 10G ports and use all the 40G native ports to create port channels between the 2 6650s. It's not the most elegant solution but seems like the best way to play within the rules and get big bandwidth between the 6610 stack and the 6650 not really a stack. The 7250s have a lot lower expectations, so I'll probably just reuse the Aruba 10G stack cabling.There is actually only 1 thing anywhere local to any of them that will even need 10G at all.
Anyway, just trying to confirm that my assumptions are correct, that the 40G ports are just that and that the breakout ports are really just logically 4 10Gs using a common connection and not really a 40Gb port. Is that correct? So I can connect 40s to 40s, and 10s, whether SFP+s or QSFP+s, together but never 40G natives to the 4x10G 40s, right?
Another question I have is in regards to the airflow direction of the 6610s. Seems most PoE switches for sale are configured as front to back (E models), and most non-PoE switches are back to front (I models). I guess this makes sense given that PoE switches would typically be closet switches and non-PoE ones ToR switches. But I guess more my question is why does does this thread say that you can't reverse the airflow in the power supply and fan modules? Are the "I" power supplies and "E" ones more different then simply the direction of the fan?
In my lab I plan to mix PoE switches (which I have some need for) and non-PoE switches which use less power as well as a pair of 6650s, all stacked in the same rack so direction needs to be the same, I don't really care which direction, just that it's all the same. Also plan to, in my office rack, run 2 PoE 6610s and 2 7250s, which is the same problem, although I really don't see there where I couldn't flip the fans on the 7250s as they are a much simpler design from what I have seen.
And one last question on this, given that the 1000W PoE power supplies and the 250W non-PoE power supplies look physically different from the back (plug on the the left vs plug on the right) I assume they are not mechanically interchangable at all. Is that correct? I also assume the 6650 uses the non-PoE 250W supply, Is that correct? And lastly does the 6610-24F also use that same 250W non-PoE supply? That switch is kind of an ugly stepchild is why I am asking, not much call for an old 1G only fiber switch, so maybe a reasonably priced donor.
In my lab I plan to mix PoE switches (which I have some need for) and non-PoE switches which use less power as well as a pair of 6650s, all stacked in the same rack so direction needs to be the same, I don't really care which direction, just that it's all the same. Also plan to, in my office rack, run 2 PoE 6610s and 2 7250s, which is the same problem, although I really don't see there where I couldn't flip the fans on the 7250s as they are a much simpler design from what I have seen.
And one last question on this, given that the 1000W PoE power supplies and the 250W non-PoE power supplies look physically different from the back (plug on the the left vs plug on the right) I assume they are not mechanically interchangable at all. Is that correct? I also assume the 6650 uses the non-PoE 250W supply, Is that correct? And lastly does the 6610-24F also use that same 250W non-PoE supply? That switch is kind of an ugly stepchild is why I am asking, not much call for an old 1G only fiber switch, so maybe a reasonably priced donor.
Well, put in a low ball best offer on (5) 6610-24F-Es that was accepted so I should have a full set of front to back 250W power supply and fan modules for the non-PoE 6610s and the bare 6650 switches, I've bought. All should show up at roughly the same time(ish).This seemed like the best place to harvest such things and I also needed rack ears for a couple of the switches I bought, so another bonus. Kinda hate to be basically buying e-waste just to turn around and really make it e-waste but 24Fs seem to almost always be "E" models and use the same 250W modules (and fans) as the almost always "I" models of any reasonably priced non-PoE models I saw. Anyway the switches, at least the pic in the listing, looked pretty clean, hopefully they all are that condition. So 2 sets of the supplies and fans will be going into a pair of 6650s, both of which are earless and fully blanked on the back. 2 different vendors, but both looked essentially unused like they were site spares, and about half the price of the next cheapest, fingers crossed. The rest are going to convert a trio of 6610-24-Is to 24-Es.
I will look at all the modules (Is & Es) to see if the fans are actually different in any significant way or just turned around to blow the other direction. Maybe I'll end up with a pile of spares. If there is some fundamental difference maybe I'll sell off a bunch of 24F-Es along with my big heaping pile of Aruba S2500s all this is replacing. Though honestly I'm not sure what real use a 24F is. It still has the same SFP+ and QSFP+ ports any other 6610 has, but all the copper ports are basically replaced with SFP ports, not SFP+, that for me would have been awesome, but just old 1G SFP ports. Maybe as a single mode campus distribution switch or something? It's the only use I can come up with in 2026 or ever actually, and that's a stretch. I certainly don't see much of an EOL turned home lab use for them. Why would you bother? If they'd made it as an all 10G 6610 that you could join to a stack that would have been a pretty cool product, but it isn't. Instead you ended up with this crippled all fiber switch and then the 6650, which is great for the sheer number of 10G ports, but it should have been stackable, but isn't.
Just received a 7250-24P. The console cable I use for 6450s and 6610s doesn't fit/work. I have a cable on order but I'd like to get started over the weekend.
I do have my existing cable with rj45 on one end, and I have a keystone. Wondering if I can cut the usb cable and connect wires on to the keystone to get things going.
If so, does anybody know which usb wires go on which rj45 wires/pins?
I do have my existing cable with rj45 on one end, and I have a keystone. Wondering if I can cut the usb cable and connect wires on to the keystone to get things going.
If so, does anybody know which usb wires go on which rj45 wires/pins?
I do not have the reference in my hand but search in this forum thread.
I made my cable for a 7250 about in 2020 by a post I found here some years ago.
Edit:
Last edited:
So far I'm not getting any response from the 7250.
Following the first link I get, with some usb wire color annotations added by me:
Pin 4 is opposite 6:
The second link is somewhat ambiguous; the usb gnd gets mapped to rj45 5,4? Is it pin 5 or pin 4 on the RJ45? Then there's a reference to the 'black' rj45 wire color which I've never come across, although color is spelled colour so perhaps it's a UK thing?
Of course I don't know whether this 7250's console port is even working.
Going back to a 6450 with the rj45 to usb cable its console immediately responds.
Following the first link I get, with some usb wire color annotations added by me:
Code:
Mini-USB Keystone (568B)
======== ===============
1 (VCC) N/C
2 Green (UART RX) 6 (Grn)
3 White (UART TX) 3 (Grn/Wht)
4 (Reserved) N/C
5 Black (GND) 4 (Blue)The second link is somewhat ambiguous; the usb gnd gets mapped to rj45 5,4? Is it pin 5 or pin 4 on the RJ45? Then there's a reference to the 'black' rj45 wire color which I've never come across, although color is spelled colour so perhaps it's a UK thing?
Of course I don't know whether this 7250's console port is even working.
Going back to a 6450 with the rj45 to usb cable its console immediately responds.
This is how it works:
Code:
USB wire | RJ45 pin
TX (WHITE) | 6
RX (GREEN) | 3
GND (BLACK) | 5This is in response to a post that has been deleted.
It's working following my previous post.
I'm not sure why brocade chose this type of usb mini connector, although it was fairly standard at the time.
You could place your argument in front of the brocade team involved with the development, and you would probably get overruled by someone who mentions that the required console cable is simply in the box together with the switch. There's a Terry Henry video where he shows what brocade provided with the different switches.
It's working following my previous post.
I'm not sure why brocade chose this type of usb mini connector, although it was fairly standard at the time.
You could place your argument in front of the brocade team involved with the development, and you would probably get overruled by someone who mentions that the required console cable is simply in the box together with the switch. There's a Terry Henry video where he shows what brocade provided with the different switches.
Last edited: