Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[metalpizza123]

Hi hi,

I still don't know what I'm doing wrong and now I'm just confused as to what's happening. As dos said, the 64xx series boots slower, but I somehow am unable to reach the bootloader. The switch is booting and functional, and I can see it on my network as an ethernet connected device(even when it's only on the management ethernet port). I can even navigate to the IP and reach the login page, but I am just so confused as to what I'm doing wrong with the serial cable. I'm still not getting output on PuTTy with the cisco console cable.

View attachment 20350

Ah frick the pinouts are different. But that explains even less about how I managed to get ANY output the first time round.

 

[fohdeesha]

Ah frick the pinouts are different. But that explains even less about how I managed to get ANY output the first time round. View attachment 20351

those are matching pinouts. transmit to recieve and vice versa. they're standard cisco style rj45 serial ports and will work with any of those adapters, if you're not getting output it usually turns out to be a driver issue under windows with all the ftdi's and related knockoffs. sometimes a reboot after the driver installs helps

 

[Cncjerry]

those are matching pinouts. transmit to recieve and vice versa. they're standard cisco style rj45 serial ports and will work with any of those adapters, if you're not getting output it usually turns out to be a driver issue under windows with all the ftdi's and related knockoffs. sometimes a reboot after the driver installs helps

Might want to try booting the ubuntu (pick your linux poison) live CD to see if that terminal works.

 

[dos]

Hi hi,

I still don't know what I'm doing wrong and now I'm just confused as to what's happening. As dos said, the 64xx series boots slower, but I somehow am unable to reach the bootloader. The switch is booting and functional, and I can see it on my network as an ethernet connected device(even when it's only on the management ethernet port). I can even navigate to the IP and reach the login page, but I am just so confused as to what I'm doing wrong with the serial cable. I'm still not getting output on PuTTy with the cisco console cable.

View attachment 20350

it might be possible whomever had the switch last changed the default baud rate to something other than 9600. can you see the boot process kick off at all with the serial cable or do you see nothing? as others said its possible that cable is toast. as long as you are using like a ft232r there shouldn't been any drivers required to install for any os. i have piles of these cables of all different types (db9, rj45, etc) and they all work. do you have another device like a cisco or some server to test? i even have some connected to my esxi servers as an additional last resort connection method for headless servers, so they are pretty versatile and should be easy to confirm functionality.

 

[metalpizza123]

it might be possible whomever had the switch last changed the default baud rate to something other than 9600. can you see the boot process kick off at all with the serial cable or do you see nothing? as others said its possible that cable is toast. as long as you are using like a ft232r there shouldn't been any drivers required to install for any os. i have piles of these cables of all different types (rs232, rj45, etc) and they all work. do you have another device like a cisco or some server to test? i even have some connected to my esxi servers as an additional last resort connection method for headless servers, so they are pretty versatile and should be easy to confirm functionality.

Hi hi quick question on whether I've just a defective unit or I'm missing something in the setup guide. I'm trying to setup the 6430-48P.

Following Fohdeesha's guide, I have the master zip downloaded and I had PuTTy set up to connect to the switch. I have a USB to serial adapter cable hooked up to the serial port, and a standard Cat 6 ethernet cable to the existing router. After booting several times with the PuTTy settings in the guide (I turned off Flow control), the output in the image showed up. After waiting several minutes, It briefly flashed some messages about loading PoE and restarted. However, since then I haven't been able to get any serial output, and can't set the values to the factory default. I've tried using the reset button on the front of the switch but to no avail. Mashing or holding the B button doesn't seem to be able to stop the bootloader, though with no output I have no clue whether it's even loading anything.

Essentially I'm just wondering if it's borked. It starts up and has a period of high fan load before slowing down, so the behaviour seems to be the same as before. The cable I'm using is a FTDI cable and I've installed their drivers for it.

Many thanks in advance.

I had some serial output to PuTTy's console, then when I rebooted no more serial output. I've tried Windows 7,10, Arch and Debian. I rebooted after installing the FTDI drivers, and tried uninstalling them too. Like this is what's so frustrating. How did I get output once, then never reach the bootloader again with the same settings? I feel mucho dumb dumb, like I'm missing something super simple.

Unfortunately, no, I don't have any other managed switches. The only one i have is an unmanaged TP link one.

 

[dos]

Im not 100% sure but:


conf t
ip access-list 22 deny 192.168.10.0/24 log
ssh access-group 22
wr mem


or

conf t
int ve 10
ip access-list 22 deny ve 10
ssh access-group 22
wr mem


or

conf t
no ip ssh client 192.168.10.0/24


Im a little hesitant to try it & lose ssh but figured I would ask here first - thanks!

i haven't had a whole bunch of time to dig into the switch config yet, but just a quick poke looks like allow ssh only on the management vrf which is probably how they indented to restrict access. in addition there is an "ip ssh client" configuration that you can specifiy specific client ips that are allowed.

 

[dos]

I had some serial output to PuTTy's console, then when I rebooted no more serial output. I've tried Windows 7,10, Arch and Debian. I rebooted after installing the FTDI drivers, and tried uninstalling them too. Like this is what's so frustrating. How did I get output once, then never reach the bootloader again with the same settings? I feel mucho dumb dumb, like I'm missing something super simple.

Unfortunately, no, I don't have any other managed switches. The only one i have is an unmanaged TP link one.

try a different cable. you might be spinning your wheels unnecessarily. i can promise you, that's how all of us ended up with so many of these cables over the years. that and aquiring extra from manufactures directly.

 

[metalpizza123]

try a different cable. you might be spinning your wheels unnecessarily. i can promise you, that's how all of us ended up with so many of these cables over the years. that and aquiring extra from manufactures directly.

Yes sir I've another cable arriving tomorrow so hopefully that fixes things.

 

[juju]

I am having some problems with dns for connected clients on my 7250.

I have a vlan 50 with a ve interface address of 10.1.50.1/24. I am connected to my pfsense box with a transit port - 10.1.2.2/30 on the switch and 10.1.2.1/30 on pfsense. I have set the following :

ip dns server-address 10.1.2.1  # pfsense transit ip
ip route 0.0.0.0/0 10.1.2.1 

# ip helper for vlan 50
ip helper-address 1 10.0.0.41

I connected my laptop to port 23 on the switch which is untagged 1/1/23 for vlan 50. The problem is, the laptop is assigned a dns server of 10.1.50.1 and there is no internet connectivity. If I manually set the dns server of the laptops connection to 10.1.2.1, everything works. I thought setting the dns server globally to 10.1.2.1 should have worked? How can I set all connected clients to have a dns server of 10.1.2.1 instead of 10.1.50.1 ?

 

[dos]

I am having some problems with dns for connected clients on my 7250.

I have a vlan 50 with a ve interface address of 10.1.50.1/24. I am connected to my pfsense box with a transit port - 10.1.2.2/30 on the switch and 10.1.2.1/30 on pfsense. I have set the following :

ip dns server-address 10.1.2.1  # pfsense transit ip
ip route 0.0.0.0/0 10.1.2.1

# ip helper for vlan 50
ip helper-address 1 10.0.0.41

I connected my laptop to port 23 on the switch which is untagged 1/1/23 for vlan 50. The problem is, the laptop is assigned a dns server of 10.1.50.1 and there is no internet connectivity. If I manually set the dns server of the laptops connection to 10.1.2.1, everything works. I thought setting the dns server globally to 10.1.2.1 should have worked? How can I set all connected clients to have a dns server of 10.1.2.1 instead of 10.1.50.1 ?

i'm going to go out on a limb here and say that is the dns server setting for the switch itself and has nothing to do with the ip address being advertised to your dhcp clients. i haven't really had much time to actually get to work with these switches since im ready to rip my hair out on how loud these damn things are despite fan swaps. in any case if this were any other brand switch then my previous comments would be correct and these appear to follow similar configuration parameters and can say that is the case here. where does your dhcp come from? set that value there and you will fix your issue.

edit:
this should be what you're after assuming the switch is your dhcp server.

ip dhcp-server pool YOUR-DHCP-POOL-NAME
  dns-server 10.1.2.1

 

[juju]

where does your dhcp come from? set that value there and you will fix your issue.

This did it for me. I was using isc-dhcp in a vm for dhcp. changing the dns option there solved my problem. I am using the dns server on pfsense now. However, I'd like to explore BIND9. I am planning to install bind9 in the same vm as the dhcp server. Can I set that as the primary dns server and the pfsense ip as the secondary without any issues?

 

[dos]

This did it for me. I was using isc-dhcp in a vm for dhcp. changing the dns option there solved my problem. I am using the dns server on pfsense now. However, I'd like to explore BIND9. I am planning to install bind9 in the same vm as the dhcp server. Can I set that as the primary dns server and the pfsense ip as the secondary without any issues?

yes. but keep in mind depending on the client, dns lookups are not sent to both servers. the first server is tried and as long as the server responds even if with an nxdomain that response will satisfy the request. only if the client doesn't get any response and the query timesout will the client query the second server. this is the behavior in windows. if you need requests to go to different servers based on the client request you need a dns server in the middle that can do conditional forwarding or recursive lookups. however, that is a bit off topic and pretty beyond the scope of this thread.

 

[juju]

So I am beginning to get my head around ACLs. Below is a snapshot of my pfsense rules for my dmz zone. I am going to shut down the dmz setup on pfsense and move it into the 7250 switch as a layer 3 subnet. So how do I setup the acls to replicate the following:

1. block dmz access to pfsense admin ports and the management network on the switch
2. block dmz access to all local subnets
3. allow exit to the internet

In acls for other subnets on the switch, I intend to create specific rules to allow access to some clients in the dmz. Also, is there an equivalent of pfsense aliases in icx switches - like I have in the screenshot above for "admin ports"?

 

[mintchipmadness]

Hi

MS24693 MACHINE SCREWS (AN507) is the generic screw type.

You will need to specify the Size and Length. They are 100 Deg Countersunk Angle. Some commercial syles have 90 Deg CS angles.

Both will work for your application.

Cad Plated

MS24693-S24

6-32

1/4

$0.05

Cad Plated

MS24693-S46

8-32

1/4

$0.06

These are from this supplier in the USA

MS24693 MACHINE SCREWS (AN507) | Aircraft Spruce


If you are in Europe.

MS24693 UNC/UNF Countersunk LAS Aerospace Ltd


Or you might try eBay.


Best of luck.


REdX1

Per a message from REdX1, these screws work on the icx7250 as well. Just in case someone else here would like to know.

 

[Blue)(Fusion]

So I am beginning to get my head around ACLs. Below is a snapshot of my pfsense rules for my dmz zone. I am going to shut down the dmz setup on pfsense and move it into the 7250 switch as a layer 3 subnet. So how do I setup the acls to replicate the following:

1. block dmz access to pfsense admin ports and the management network on the switch
2. block dmz access to all local subnets
3. allow exit to the internet

View attachment 20378


In acls for other subnets on the switch, I intend to create specific rules to allow access to some clients in the dmz. Also, is there an equivalent of pfsense aliases in icx switches - like I have in the screenshot above for "admin ports"?

There are no aliases that I know of in the switch ACLs.

Here's an example you should be able to work from. This is from 8.0.30 firmware not sure if it's the same on 7xxx series firmware.

ip access-list extended guest-v4
 remark DENY ADMIN ACCESS TO SWITCH
deny tcp any host 10.23.15.1 eq ssh log
deny tcp any host 10.23.15.1 eq telnet log
deny tcp any host 10.23.15.1 eq http log
deny tcp any host 10.23.15.1 eq ssl log
remark ALLOW SAME VLAN TRAFFIC
permit ip any 10.23.15.0 0.0.0.255
remark ALLOW DHCP
permit udp any any eq bootps
permit udp any any eq bootpc
remark ALLOW ICMP
permit icmp any any
remark ALLOW ESTABLISHED TCP TRAFFIC
permit tcp any any established
remark ALLOW DNS REQUESTS
permit udp any host 10.23.60.3 eq dns
permit tcp any host 10.23.60.3 eq dns
permit udp any host 10.23.60.5 eq dns
permit tcp any host 10.23.60.5 eq dns
remark ALLOW ACCESS TO CACHE01 HTTP/S
permit tcp any host 10.23.60.5 eq http
permit tcp any host 10.23.60.5 eq ssl
remark ALLOW RETURN OF SNMP TRAFFIC TO MON01
permit udp any eq snmp host 10.23.60.81
permit udp any eq snmp-trap host 10.23.60.81
remark DENY ALL OTHER INTER-VLAN TRAFFIC
deny ip any 10.0.0.0 0.255.255.255 log
remark PERMIT REMAINING TRAFFIC
permit ip any any
enable-accounting

In the above example, you will see same-VLAN traffic is permitted while other VLAN traffic can not be sent with a few exceptions (already established TCP from another VLAN, some specific TCP/UDP traffic (remember UDP traffic is never "established"). After being denied access to other VLANs (less the specific allows), all traffic is permitted (to the internet).

 

[dos]

So I am beginning to get my head around ACLs. Below is a snapshot of my pfsense rules for my dmz zone. I am going to shut down the dmz setup on pfsense and move it into the 7250 switch as a layer 3 subnet. So how do I setup the acls to replicate the following:

1. block dmz access to pfsense admin ports and the management network on the switch
2. block dmz access to all local subnets
3. allow exit to the internet

View attachment 20378


In acls for other subnets on the switch, I intend to create specific rules to allow access to some clients in the dmz. Also, is there an equivalent of pfsense aliases in icx switches - like I have in the screenshot above for "admin ports"?

just keep in mind that even though this is a layer 3 switch, it is not an edge router / firewall. unlike nexus switches, these don't support any type of nat, so if you plan to connect this to the internet you will still need something upstream to perfrom snat and dnat.

 

[juju]

There are no aliases that I know of in the switch ACLs.

Here's an example you should be able to work from. This is from 8.0.30 firmware not sure if it's the same on 7xxx series firmware.

ip access-list extended guest-v4
remark DENY ADMIN ACCESS TO SWITCH
deny tcp any host 10.23.15.1 eq ssh log
deny tcp any host 10.23.15.1 eq telnet log
deny tcp any host 10.23.15.1 eq http log
deny tcp any host 10.23.15.1 eq ssl log
remark ALLOW SAME VLAN TRAFFIC
permit ip any 10.23.15.0 0.0.0.255
remark ALLOW DHCP
permit udp any any eq bootps
permit udp any any eq bootpc
remark ALLOW ICMP
permit icmp any any
remark ALLOW ESTABLISHED TCP TRAFFIC
permit tcp any any established
remark ALLOW DNS REQUESTS
permit udp any host 10.23.60.3 eq dns
permit tcp any host 10.23.60.3 eq dns
permit udp any host 10.23.60.5 eq dns
permit tcp any host 10.23.60.5 eq dns
remark ALLOW ACCESS TO CACHE01 HTTP/S
permit tcp any host 10.23.60.5 eq http
permit tcp any host 10.23.60.5 eq ssl
remark ALLOW RETURN OF SNMP TRAFFIC TO MON01
permit udp any eq snmp host 10.23.60.81
permit udp any eq snmp-trap host 10.23.60.81
remark DENY ALL OTHER INTER-VLAN TRAFFIC
deny ip any 10.0.0.0 0.255.255.255 log
remark PERMIT REMAINING TRAFFIC
permit ip any any
enable-accounting

In the above example, you will see same-VLAN traffic is permitted while other VLAN traffic can not be sent with a few exceptions (already established TCP from another VLAN, some specific TCP/UDP traffic (remember UDP traffic is never "established"). After being denied access to other VLANs (less the specific allows), all traffic is permitted (to the internet).

@Blue)(Fusion thanks for this. Will get me started. 3 questions:

1. Does this allow incoming traffic from another vlan? e.g. I will have a printer sitting in this dmz. Does this restrict access to that printer from other vlans on the network?
2. The devices in the dmz wont need to talk to each other in the same vlan - e.g. printer and sprinkler system. so remove the rule for same vlan traffic?
3. when applying to the dmz ve interface, is this for inbound or outbound packets? - ip access-group "dmz acl" in/out

 

[Blue)(Fusion]

@Blue)(Fusion thanks for this. Will get me started. 3 questions:

1. Does this allow incoming traffic from another vlan? e.g. I will have a printer sitting in this dmz. Does this restrict access to that printer from other vlans on the network?
2. The devices in the dmz wont need to talk to each other in the same vlan - e.g. printer and sprinkler system. so remove the rule for same vlan traffic?
3. when applying to the dmz ve interface, is this for inbound or outbound packets? - ip access-group "dmz acl" in/out

1. No*. This ACL is meant to be applied as inbound on the VE. In the example ACL, established TCP traffic is permitted to another VLAN that initiated the traffic, but UDP must specifically be allowed on a host, network, and/or port basis. Note in the example ACL, the return path of SNMP requests from devices on the guest network to my LibreNMS server on a different network.
2. Don't quote me but I believe you can modify the rule to allow traffic to the VE gateway IP and traffic bound to other devices on the L2 network will be dropped.
3. Inbound. It's almost always preferable to do inbound ACLs.

 

[fohdeesha]

Still have one ICX7250-48 left for sale, cheapest one I see for sale on ebay without port damage is around 260+40 shipping, I could do $240 including a couple optics or 10gbase-T modules. pickup only tho in northern indianapolis, NO SHIP

some ICX6610s too

 

[custom90gt]

Still have one ICX7250-48 left for sale, cheapest one I see for sale on ebay without port damage is around 260+40 shipping, I could do $240 including a couple optics or 10gbase-T modules. pickup only tho in northern indianapolis, NO SHIP

some ICX6610s too

How much for a 6610? I am enjoying my 6450, but the temptation is there to go with 10GB, lol.