Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[neb50]

The router just needs to be connected to one of the VLAN's and your setup looks like it should be VLAN 2. The other VLAN's don't need to go to the router unless you are wanting it to do the routing. You can either do that by setting up the UDMP to use port 9 on VLAN2 and keep it tagged on the switch or untag it on switch and have the UDMP use the untagged port 9.

The DHCP server in your UDMP can have the gateway for each subnet point to the ip of each of the ve interfaces (10.0.x.2) and the switch will figure the rest out. In the Edgerouter it is listed as Router in the DHCP web setup to configure the gateway.

 

[losx]

So the problem I am running into is if I take vlan 40 for example and remove the tag to e 1/2/1 which is on vlan 2 I can no longer ping anything outside the switch. Within the switch I can access other vlans but I lose access to the anything outside the switch and outside the switch I can't get back in either.

Is something being done incorrectly?

vlan 40 name TestVlan40Network by port
untagged ethe 1/1/25
router-interface ve 40

interface ve 40
ip address 10.0.40.2 255.255.255.0
ip helper-address 1 10.0.40.1

 

[LodeRunner]

Hi All

So bought this switch ICX7150-48ZP, got it for 500$ including shipping to Denmark so you guys in US might be able to get it cheaper.

Ruckus ICX7150-48ZP-E2X10G Z Series ICX 7150-48ZP Switch *Read Description* | eBay

For sale is a Ruckus ICX7150-48ZP-E2X10G. Local pick up is available, and always free. The mode lights are not working but status light still works. Mode/reset buttons do not function either. 1x RJ45 Console Cable.

www.ebay.com

After updating firmware to 8.0.80f the front mode button was working.
it came with 8.0.6 something

/CC @fohdeesha as an answer for the PM I sent you


Also should i run 8.0.9xx or 08.0.8xxx

I came so close to pulling the trigger on one of those but the description of damage left me unsure. I should have damned well asked and then best-offered them lower than that $750. I am planning on getting R720 or R750s to replace UAP-ACPros and the multigig would have been nice. But I don't have any wireless clients (yet) that can even approach that speed.

 

[neb50]

So the problem I am running into is if I take vlan 40 for example and remove the tag to e 1/2/1 which is on vlan 2 I can no longer ping anything outside the switch. Within the switch I can access other vlans but I lose access to the anything outside the switch and outside the switch I can't get back in either.

Is something being done incorrectly?

vlan 40 name TestVlan40Network by port
untagged ethe 1/1/25
router-interface ve 40

interface ve 40
ip address 10.0.40.2 255.255.255.0
ip helper-address 1 10.0.40.1

The helper-address should point to your router/DHCP server ip. The ip for stuff in that VLAN will be 10.0.40.x, mask of 255.255.255.0, and gateway set to the ip of the VLAN ve ip.

Is VLAN2 untagged on the router port? And can it connect out to the router?

 

[itronin]

Edit : I meant 192.168.1/24 below in the question.

Is something being done incorrectly?

so for sure and IMO you've done something I consider unorthodox. You've configed 1/2/1 as dual-mode 2 <== that's vlan id 2
and configured 1/2/1 as a tagged member of vlan 2.

A couple of suggestions first and others may differ:
simpler is better.
Don't shoot for the moon on your first config. Get the basics working correctly and then augment/add on to it.
Try and maintain configuration parity / purpose between the devices in your network stack. I say this because you mentioned spinning up VLAN 2 in your router/firewall and different IP's I'm not sure as to the purpose.
For example if you are using specific vlans in the swtich but not in the router don't re-use them in the router. It may get confusing...
Double check your router configuration. You've provided a puzzle - but only have the pieces. I'm not saying post the router config ... just make sure you don't have a typo in there.
tag your trunks, don't mix untagged and tagged unles you ABSOLUTELY have to do so.
simple devices should be untagged.
Routed connections can be untagged - IMO either use all tagged (with sub interfaces) or untagged (simpler) for routed connections. Make a mistake with tag/untag at the same and everything stops working or behaves oddly.


Observations:
Yes, something is off. you should not have to make port 1/2/1 a tagged member of all those vlans.
If you get one VLAN to work correctly then they will all work correctly.

Question:
You aren't using 192.168.1/24 anywhere else? It is only being used for the transit VLAN between your switch and your router?
if you are I recommend changing the subnet on your transit vlan to something that is not used anywhere else.

Reminder:

In order for a VE to come up you need at least one active port that is a member of that VLAN.

troubleshooting suggestions:

save yoru config, strip it down to basics, and start with say the transit vlan.
Assuming you have NAT properly configured the switch IP should be able to ping out/traceroute out to the Internetwebs.
Don't use dual-mode on your transit port, just make it an untagged member of that vlan.
Once that works add another vlan, get it working with a static IP first.
Once that works add your DHCP helper and make sure as stated before that the def gw is the VE interface for that subnet and the helper IP is the IP of your DHCP server (which appears to be your router/firewall).
After your first access vlan is working just clone the config, changig out the def gw in the DHCP scope for the appropriate subnet of that vlan

 

[losx]

This is the config after I pared it down quite a bit... I started as a layer 2 switch with multiple vlans and that worked fine (router doing routing). Then the posts said setup intervlan routing and remove the internet entirely and I got that to work... when I started to add the transit gateway and DHCP I screwed it all up. I will try to go to other direction first like you mentioned let me go bang my head against this some more and read up more... the config is all screwy now because I have been trying multiple things and I am likely just making it worse. I think I will just do 3 vans... transit 20 and 40 for now... lets see how this goes

 

[losx]

So I played with it for a while and it I think the issue is with the UDMP managing the DHCP portion... The only way to do it is to create a new network on the UDMP which doesn't seem right...

 

[itronin]

I don't have any experience with UBQ so I'm trying to draw parallels with other solutions.

You mentioned you had this all working with the switch as L2 so the UDMP was handling all routing and dhcp. If you were using vlans then UDMP had an interface configured for *and* connected to each vlan (either tagged VLAN or physical).

The concept for DHCP helper (relay) is that you need a DHCP server that can create a DHCP scope for each IP subnet that exists on your switch. The scopes need to be created without an associated IP'ed interface on the DHCP server (in your case the UDMP),

It may be that you have to define the network which in essence creates the scope, but can you do it without an associated UDMP network interface in that scope's subnet? I don't know the answer to that question.

If the router has a directly connected interface to each vlan then you don't even need IP helper specified! Its already listening on an interface that can respond to DHCP Then though you'll end up with split path routing for your Internet traffic.

Some router/firewalls do NOT support defining standalone DHCP scopes. For example PFsense comes to mind.
A quick search shows discussions of folks configuring UBQ devices with dhcp relay pointing to no UBQ DHCP servers but I did not see anything about using UDMP as a DHCP server with standalone scopes. Doesn't mean you can't, just I didn't find it with a quick search.

Simply defining the network in the UDMP could be necessary necessary to instantiate the DHCP scope and it may also be required to set up the appropriate NAT for Internet destined traffic originating from the defined network. Again, I don't know. just speculating.

btw I think you have a typo in the first screenshot where you define the gateway IP - as stated previously that will need to be the switch's VE for the defined subnet. From your config above: 10.0.40.2/24 is that I think goes in the GW field in the first screenshot.

If you have a windows server on your network, add the DHCP role to it and use that for DHCP server. You could also set up a linux/*BSD server too and run maybe ISC DHCP. However that adds additional complexity and dependencies to your network infrastructure that perhaps you are trying to avoid. Therein lies the rub.

btw, nothing wrong with doing things at 2 and 3am. Carpe Noctem!

 

[nerdalertdk]

I came so close to pulling the trigger on one of those but the description of damage left me unsure. I should have damned well asked and then best-offered them lower than that $750. I am planning on getting R720 or R750s to replace UAP-ACPros and the multigig would have been nice. But I don't have any wireless clients (yet) that can even approach that speed.

I was also quite nervous since I also paid for EU shipping and taxes, but they wrote they gave full return if anything else failed

plus found an forum post on ruckus site saying the front button only works from X firmware version

I needed 8 x 10g ports so it’s the only options in the 7150 line

 

[Vesalius]

I was also quite nervous since I also paid for EU shipping and taxes, but they wrote they gave full return if anything else failed

plus found an forum post on ruckus site saying the front button only works from X firmware version

I needed 8 x 10g ports so it’s the only options in the 7150 line

Thanks to your post I snagged what might have been the last one for $400 shipped in the US. Thanks!!

 

[nerdalertdk]

Thanks to your post I snagged what might have been the last one for $400 shipped in the US. Thanks!!

Hehe did see that last one was gone and was wondering who of you guys took it since they have been on eBay for a while.

400$ that’s a still for that switch.
I’m in the process of finding some quiet fans for it but it’s not that easy it seams.

it’s a bit louder then my edge switch

 

[Roelf Zomerman]

in the Unifi you can create a new network, this by giving an interface (or VLAN on an interface) an ip address and subnet (for example 10.0.40.1/24) and then the system automatically created the DHCP config for it.. but you dont have to accept that default config.. you can either disable DHCP completely or change the DHCP Gateway IP address by selecting manual ..

that way you can still route everything through your switch if you wanted to.. and the switch would basically have another route of 0.0.0.0/0 --> 10.0.40.1.. for internet access..

 

[Roelf Zomerman]

I changed the fans on my 6450 as the old ones where getting annoyingly loud..
I ordered the EFB0412VHD-F00 ones as suggested in this (very long ) thread.. but now the switch is acting up.. it is really quiet for most of the time (fans running at low speed) and all of a sudden the switch goes bezerk and thinks it needs to run the fans for ~5 minutes at full throttle.. before going quiet again..

anyone with the same problem?

 

[Scarlet]

If your fans are spinning up then the 6450 crossed the temperature threshold (64 deg C) to fan speed level 2 (full speed). The fans keep spinning at that higher speed until the switch reaches the low threshhold (59 deg C) where the fans will spin down do speed level 1 again.

This means that the fans cannot keep your switch cool enough - at your ambient temperature - to keep the fans constantly at speed level 1. You could lower your ambient temperature (e.g. by actively exhausting air from a rack) or use fans that pull more air through the switch.

I'm using two Sunon KDE1204PKV3 that have even less air flow (6.8 CFM) than your EFB0412VHD-F00 (10.1 CFM), so I'm guessing your ambient temperatures are quite high. I only had the spinning up sometimes in summer when the outside temperature was well above 30 deg C.

 

[losx]

I want to thank everyone for the help. I set it all up minus the DHCP server and the ip helper and it worked. If I assigned a static IP and did traceroute etc I saw everything work fine. Inter vlan routing works and going out to the web. The second I would add a network in the unifi gateway with a DHCP gateway pointing to the switch 10.0.40.2 (no ip helper address yet) it screwed everything up. I could no longer ping to the outside world from the switch and I couldn't even do inter vlan routing (I'm fairly sure I am remembering this correctly) with the same static address.

Seems like when I define a network the UDMP acts as the gateway and creates an interface with the gateway address specified so it does not seem like it can work as a standalone DHCP server.

Put a message on unifi boards to confirm... we will se when I get a response.

Digging through udm os it leverages dnsmasq which I believe can do standalone dhcp scopes?

 

[neb50]

This is how you do it with an Edgerouter POE from the Services/DHCP settings. You just create a new DHCP range for each VLAN. There is not an "interface" or anything else configured for the VLAN's on the router except for one static route to the subnet's of the VLAN's on the port that the switch is connected to.

Do you have the ability to just configure the DHCP server settings on UDMP like this or do you have to use an interface to do it through a "wizard" tool?

 

[LodeRunner]

Apparently I have gone insane because same thing that wasn't working last night works this morning, and rerunning the flash commands on primary and secondary, which I swore I did, resulted in proper boot code updates.

 

[losx]

@neb50 I posted the screens above your post... unfortunately I have to create a network which seems to create and interface and DHCP at the same time. I have no way to just do DHCP it seems

 

[ArmedAviator]

@losx

This is the unfortunate problem of the fool-proofing of electronics and power-users wanting to use them. The DHCP server no longer works as designed by RFC. Instead, you end up with the pfSense, OPNsense, Unifi/EdgeOS/RouterOS/etc. limitations. They've given me issues for years and after spending 2 hours of setting up BIND named and ISC-DHCP, I am infinitely more happy with things running as expected/designed. Spend the 2 hours setting up a DHCP/DNS virtual machine of your choice and forget about the fool-proofed hassle of non-compliant DHCP servers.

 

[Fallen Kell]

@losx

This is the unfortunate problem of the fool-proofing of electronics and power-users wanting to use them. The DHCP server no longer works as designed by RFC. Instead, you end up with the pfSense, OPNsense, Unifi/EdgeOS/RouterOS/etc. limitations. They've given me issues for years and after spending 2 hours of setting up BIND named and ISC-DHCP, I am infinitely more happy with things running as expected/designed. Spend the 2 hours setting up a DHCP/DNS virtual machine of your choice and forget about the fool-proofed hassle of non-compliant DHCP servers.

Yeah, I have been debating this myself. I will probably be doing this next weekend (I am just debating on if I should use the existing FreeNAS VM I have already and set it up in there (cons being that I don't think it saves the config on updates, so I would need to do that manually and restore each time), or just bit the bullet and configure another VM just for DNS/DHCP (maybe rsyslog as well)).