Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[dos]

I am having some problems with dns for connected clients on my 7250.

I have a vlan 50 with a ve interface address of 10.1.50.1/24. I am connected to my pfsense box with a transit port - 10.1.2.2/30 on the switch and 10.1.2.1/30 on pfsense. I have set the following :

ip dns server-address 10.1.2.1  # pfsense transit ip
ip route 0.0.0.0/0 10.1.2.1

# ip helper for vlan 50
ip helper-address 1 10.0.0.41

I connected my laptop to port 23 on the switch which is untagged 1/1/23 for vlan 50. The problem is, the laptop is assigned a dns server of 10.1.50.1 and there is no internet connectivity. If I manually set the dns server of the laptops connection to 10.1.2.1, everything works. I thought setting the dns server globally to 10.1.2.1 should have worked? How can I set all connected clients to have a dns server of 10.1.2.1 instead of 10.1.50.1 ?

i'm going to go out on a limb here and say that is the dns server setting for the switch itself and has nothing to do with the ip address being advertised to your dhcp clients. i haven't really had much time to actually get to work with these switches since im ready to rip my hair out on how loud these damn things are despite fan swaps. in any case if this were any other brand switch then my previous comments would be correct and these appear to follow similar configuration parameters and can say that is the case here. where does your dhcp come from? set that value there and you will fix your issue.

edit:
this should be what you're after assuming the switch is your dhcp server.

ip dhcp-server pool YOUR-DHCP-POOL-NAME
  dns-server 10.1.2.1

 

[juju]

where does your dhcp come from? set that value there and you will fix your issue.

This did it for me. I was using isc-dhcp in a vm for dhcp. changing the dns option there solved my problem. I am using the dns server on pfsense now. However, I'd like to explore BIND9. I am planning to install bind9 in the same vm as the dhcp server. Can I set that as the primary dns server and the pfsense ip as the secondary without any issues?

 

[dos]

This did it for me. I was using isc-dhcp in a vm for dhcp. changing the dns option there solved my problem. I am using the dns server on pfsense now. However, I'd like to explore BIND9. I am planning to install bind9 in the same vm as the dhcp server. Can I set that as the primary dns server and the pfsense ip as the secondary without any issues?

yes. but keep in mind depending on the client, dns lookups are not sent to both servers. the first server is tried and as long as the server responds even if with an nxdomain that response will satisfy the request. only if the client doesn't get any response and the query timesout will the client query the second server. this is the behavior in windows. if you need requests to go to different servers based on the client request you need a dns server in the middle that can do conditional forwarding or recursive lookups. however, that is a bit off topic and pretty beyond the scope of this thread.

 

[juju]

So I am beginning to get my head around ACLs. Below is a snapshot of my pfsense rules for my dmz zone. I am going to shut down the dmz setup on pfsense and move it into the 7250 switch as a layer 3 subnet. So how do I setup the acls to replicate the following:

1. block dmz access to pfsense admin ports and the management network on the switch
2. block dmz access to all local subnets
3. allow exit to the internet

In acls for other subnets on the switch, I intend to create specific rules to allow access to some clients in the dmz. Also, is there an equivalent of pfsense aliases in icx switches - like I have in the screenshot above for "admin ports"?

 

[mintchipmadness]

Hi

MS24693 MACHINE SCREWS (AN507) is the generic screw type.

You will need to specify the Size and Length. They are 100 Deg Countersunk Angle. Some commercial syles have 90 Deg CS angles.

Both will work for your application.

Cad Plated

MS24693-S24

6-32

1/4

$0.05

Cad Plated

MS24693-S46

8-32

1/4

$0.06

These are from this supplier in the USA

MS24693 MACHINE SCREWS (AN507) | Aircraft Spruce


If you are in Europe.

MS24693 UNC/UNF Countersunk LAS Aerospace Ltd


Or you might try eBay.


Best of luck.


REdX1

Per a message from REdX1, these screws work on the icx7250 as well. Just in case someone else here would like to know.

 

[Blue)(Fusion]

So I am beginning to get my head around ACLs. Below is a snapshot of my pfsense rules for my dmz zone. I am going to shut down the dmz setup on pfsense and move it into the 7250 switch as a layer 3 subnet. So how do I setup the acls to replicate the following:

1. block dmz access to pfsense admin ports and the management network on the switch
2. block dmz access to all local subnets
3. allow exit to the internet

View attachment 20378


In acls for other subnets on the switch, I intend to create specific rules to allow access to some clients in the dmz. Also, is there an equivalent of pfsense aliases in icx switches - like I have in the screenshot above for "admin ports"?

There are no aliases that I know of in the switch ACLs.

Here's an example you should be able to work from. This is from 8.0.30 firmware not sure if it's the same on 7xxx series firmware.

ip access-list extended guest-v4
 remark DENY ADMIN ACCESS TO SWITCH
deny tcp any host 10.23.15.1 eq ssh log
deny tcp any host 10.23.15.1 eq telnet log
deny tcp any host 10.23.15.1 eq http log
deny tcp any host 10.23.15.1 eq ssl log
remark ALLOW SAME VLAN TRAFFIC
permit ip any 10.23.15.0 0.0.0.255
remark ALLOW DHCP
permit udp any any eq bootps
permit udp any any eq bootpc
remark ALLOW ICMP
permit icmp any any
remark ALLOW ESTABLISHED TCP TRAFFIC
permit tcp any any established
remark ALLOW DNS REQUESTS
permit udp any host 10.23.60.3 eq dns
permit tcp any host 10.23.60.3 eq dns
permit udp any host 10.23.60.5 eq dns
permit tcp any host 10.23.60.5 eq dns
remark ALLOW ACCESS TO CACHE01 HTTP/S
permit tcp any host 10.23.60.5 eq http
permit tcp any host 10.23.60.5 eq ssl
remark ALLOW RETURN OF SNMP TRAFFIC TO MON01
permit udp any eq snmp host 10.23.60.81
permit udp any eq snmp-trap host 10.23.60.81
remark DENY ALL OTHER INTER-VLAN TRAFFIC
deny ip any 10.0.0.0 0.255.255.255 log
remark PERMIT REMAINING TRAFFIC
permit ip any any
enable-accounting

In the above example, you will see same-VLAN traffic is permitted while other VLAN traffic can not be sent with a few exceptions (already established TCP from another VLAN, some specific TCP/UDP traffic (remember UDP traffic is never "established"). After being denied access to other VLANs (less the specific allows), all traffic is permitted (to the internet).

 

[dos]

So I am beginning to get my head around ACLs. Below is a snapshot of my pfsense rules for my dmz zone. I am going to shut down the dmz setup on pfsense and move it into the 7250 switch as a layer 3 subnet. So how do I setup the acls to replicate the following:

1. block dmz access to pfsense admin ports and the management network on the switch
2. block dmz access to all local subnets
3. allow exit to the internet

View attachment 20378


In acls for other subnets on the switch, I intend to create specific rules to allow access to some clients in the dmz. Also, is there an equivalent of pfsense aliases in icx switches - like I have in the screenshot above for "admin ports"?

just keep in mind that even though this is a layer 3 switch, it is not an edge router / firewall. unlike nexus switches, these don't support any type of nat, so if you plan to connect this to the internet you will still need something upstream to perfrom snat and dnat.

 

[juju]

There are no aliases that I know of in the switch ACLs.

Here's an example you should be able to work from. This is from 8.0.30 firmware not sure if it's the same on 7xxx series firmware.

ip access-list extended guest-v4
remark DENY ADMIN ACCESS TO SWITCH
deny tcp any host 10.23.15.1 eq ssh log
deny tcp any host 10.23.15.1 eq telnet log
deny tcp any host 10.23.15.1 eq http log
deny tcp any host 10.23.15.1 eq ssl log
remark ALLOW SAME VLAN TRAFFIC
permit ip any 10.23.15.0 0.0.0.255
remark ALLOW DHCP
permit udp any any eq bootps
permit udp any any eq bootpc
remark ALLOW ICMP
permit icmp any any
remark ALLOW ESTABLISHED TCP TRAFFIC
permit tcp any any established
remark ALLOW DNS REQUESTS
permit udp any host 10.23.60.3 eq dns
permit tcp any host 10.23.60.3 eq dns
permit udp any host 10.23.60.5 eq dns
permit tcp any host 10.23.60.5 eq dns
remark ALLOW ACCESS TO CACHE01 HTTP/S
permit tcp any host 10.23.60.5 eq http
permit tcp any host 10.23.60.5 eq ssl
remark ALLOW RETURN OF SNMP TRAFFIC TO MON01
permit udp any eq snmp host 10.23.60.81
permit udp any eq snmp-trap host 10.23.60.81
remark DENY ALL OTHER INTER-VLAN TRAFFIC
deny ip any 10.0.0.0 0.255.255.255 log
remark PERMIT REMAINING TRAFFIC
permit ip any any
enable-accounting

In the above example, you will see same-VLAN traffic is permitted while other VLAN traffic can not be sent with a few exceptions (already established TCP from another VLAN, some specific TCP/UDP traffic (remember UDP traffic is never "established"). After being denied access to other VLANs (less the specific allows), all traffic is permitted (to the internet).

@Blue)(Fusion thanks for this. Will get me started. 3 questions:

1. Does this allow incoming traffic from another vlan? e.g. I will have a printer sitting in this dmz. Does this restrict access to that printer from other vlans on the network?
2. The devices in the dmz wont need to talk to each other in the same vlan - e.g. printer and sprinkler system. so remove the rule for same vlan traffic?
3. when applying to the dmz ve interface, is this for inbound or outbound packets? - ip access-group "dmz acl" in/out

 

[Blue)(Fusion]

@Blue)(Fusion thanks for this. Will get me started. 3 questions:

1. Does this allow incoming traffic from another vlan? e.g. I will have a printer sitting in this dmz. Does this restrict access to that printer from other vlans on the network?
2. The devices in the dmz wont need to talk to each other in the same vlan - e.g. printer and sprinkler system. so remove the rule for same vlan traffic?
3. when applying to the dmz ve interface, is this for inbound or outbound packets? - ip access-group "dmz acl" in/out

1. No*. This ACL is meant to be applied as inbound on the VE. In the example ACL, established TCP traffic is permitted to another VLAN that initiated the traffic, but UDP must specifically be allowed on a host, network, and/or port basis. Note in the example ACL, the return path of SNMP requests from devices on the guest network to my LibreNMS server on a different network.
2. Don't quote me but I believe you can modify the rule to allow traffic to the VE gateway IP and traffic bound to other devices on the L2 network will be dropped.
3. Inbound. It's almost always preferable to do inbound ACLs.

 

[fohdeesha]

Still have one ICX7250-48 left for sale, cheapest one I see for sale on ebay without port damage is around 260+40 shipping, I could do $240 including a couple optics or 10gbase-T modules. pickup only tho in northern indianapolis, NO SHIP

some ICX6610s too

 

[custom90gt]

Still have one ICX7250-48 left for sale, cheapest one I see for sale on ebay without port damage is around 260+40 shipping, I could do $240 including a couple optics or 10gbase-T modules. pickup only tho in northern indianapolis, NO SHIP

some ICX6610s too

How much for a 6610? I am enjoying my 6450, but the temptation is there to go with 10GB, lol.

 

[fohdeesha]

How much for a 6610? I am enjoying my 6450, but the temptation is there to go with 10GB, lol.

prolly around whatever ebay prices are currently minus ~20% - can you do indy pickup?

 

[Blue)(Fusion]

I also have a 6610-48-PE dual PSU available a few hours east of fohdeesha.

 

[custom90gt]

prolly around whatever ebay prices are currently minus ~20% - can you do indy pickup?

Ahh bummer, didn't see the local pickup piece.

 

[tjk]

Any recommendations for alternatives to the ICX7750-48F's? I am using these as TOR's, but they are now very hard to find on the used market.

Not sure what Arista model would be the equiv that can still run latest code?

Don't need any layer3 features, all layer2 use case, trunking, vlans, etc.

Thanks!

 

[deeceesth]

I have a couple 7250s that came with the L2 only SPS firmware. Initially I ran them as is since the FW was new enough and I was just dipping my toes into the brocade world.

Since then I've flashed them up to 80.0.90 with the L3 SPR firmware. I don't use/need L3 stuff and the guides say it should work fine for L2 only if you don't configure any L3 features.

I have been experiencing strange network issues (can't access anything on my local network or through my firewall for a few minutes at a time) and I suspect it could be the switch with SPR/L3 firmware since they started around the time I did the upgrade. I don't have any evidence other than the timing. Another strange issue is that I can no longer SSH or access the webui (i dont use the webui but it was working in L2 mode) of the switch after flashing the SPR variant. I followed the parts of the guide (as of nov 5) that talk about enabling SSH and the webui and it still doesn't work. I also noticed that the switch doesn't always grab a DHCP lease from my pfsense firewall DHCP server.

Before I go and reflash to L2 fw for diagnostic purposes I thought I'd ask here if anyone has similar experiences or just general knowledge about using these switches in L3 vs L2 mode.
Is there any way that using the L3 firmware could cause issues if I'm trying to use it for L2 only? Is there anything in the guide that could cause some L3 features to be turned on?

 

[LodeRunner]

I'd start by setting a static IP on your management interface. DHCP reservations are usually fine, but I would avoid DHCP for infrastructure. If that doesn't sort it out, backup the config and factory default it.

 

[xzy]

I purchased a Brocade ICX 6450-48 and updated the config as well as setting it up via the docs provided on the first page of the forum.

Due to me finding the stock fans too loud I modified the switch and changed them for the Delta EFB0412VHD-F00 which were advised by another person in this forum.It is a little bit quiter than the stock ones but it's still loud.

I have another fan of the same model.Is it worth using another fan as well or is this expected?

 

[deeceesth]

I'd start by setting a static IP on your management interface. DHCP reservations are usually fine, but I would avoid DHCP for infrastructure. If that doesn't sort it out, backup the config and factory default it.

Yea good point, I have been meaning to do that. I'll try that out before resorting to flashing.

 

[deeceesth]

Update on my network interruption issue.

I set all my infrastructure to static IPs instead of DHCP reservations (7250 48P, 2xR710, and 1xR610). I was still getting the interruptions.
I flashed a 7150 with SPS fw and swapped out the 7250. It appears to be more stable now no interruptions yet. It's very noticeable so I'll know in a day or two if it for sure fixes the issue.

In the mean time I reset the 7250 to factory (still on spr) and manually set it up again. going to swap that back in if the 7150 with sps reproduces the issue.