Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

fohdeesha

Kaini Industries
Nov 20, 2016
2,726
3,066
113
33
fohdeesha.com
Are you just trying to relay IPv6 DHCP to an actual ipv6 DHCP server? that's doable on 8030:

Code:
interface ve 10
 ip address 192.168.1.1 255.255.255.0
 ip helper-address 1 172.16.110.2
 ipv6 address 2001:470:c15e:8000::1/64
 ipv6 enable
 ipv6 dhcp-relay destination 2001:470:c15e::2
 ipv6 dhcp-relay include-options interface-id remote-id
 ipv6 nd managed-config-flag
if you want to do prefix delegation from the ICX itself, yeah that's gunna be an issue
 

kache

New Member
Jun 27, 2020
15
2
3
Are you just trying to relay IPv6 DHCP to an actual ipv6 DHCP server? that's doable on 8030:



Code:
interface ve 10

ip address 192.168.1.1 255.255.255.0

ip helper-address 1 172.16.110.2

ipv6 address 2001:470:c15e:8000::1/64

ipv6 enable

ipv6 dhcp-relay destination 2001:470:c15e::2

ipv6 dhcp-relay include-options interface-id remote-id

ipv6 nd managed-config-flag


if you want to do prefix delegation from the ICX itself, yeah that's gunna be an issue


It doesn't seem to be possible even with SLAAC though:

Brocade:

Code:
interface ve 1
 ip address 192.168.2.254 255.255.255.0
 ipv6 address some:thing:else:3901::2/64
 ipv6 enable
!
interface ve 10
 ip address 192.168.10.1 255.255.255.0
 ip helper-address 1 192.168.10.31
 ip helper-address 2 192.168.10.21
 ipv6 address some:thing:else:3902::2/64
 ipv6 enable
 ipv6 dhcp-relay destination some:thing:else:3901::2
 ipv6 dhcp-relay include-options interface-id remote-id
 ipv6 nd managed-config-flag
!
interface ve 20
 ip address 192.168.20.1 255.255.255.0
 ip helper-address 1 192.168.10.31
 ip helper-address 2 192.168.10.21
 ipv6 address some:thing:else:3903::3/64
 ipv6 enable
 ipv6 dhcp-relay destination some:thing:else:3901::2
 ipv6 dhcp-relay include-options interface-id remote-id
 ipv6 nd managed-config-flag


Edgerouter:

Code:
ubnt@EdgeRouter-Pro-8-Port# show
 no-dns
 pd 0 {
     interface eth1 {
         host-address :1
         no-dns
         prefix-id 1
         service slaac
     }
     prefix-length 56
 }
 prefix-only
 rapid-commit enable

The problem is that the edgerouter is only assigning a /64 to the interface so it doesn't identify the presence of the other subnets on the other side:
Code:
ubnt@EdgeRouter-Pro-8-Port:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address                        S/L  Description
---------    ----------                        ---  -----------
eth0         -                                 u/u  Internet (PPPoE)
eth1         192.168.2.1/24                    u/u  Local
             some:thing:else:3901::1/64
eth2         -                                 u/D
eth3         -                                 u/D
eth4         -                                 u/D
eth5         -                                 u/D
eth6         -                                 u/D
eth7         -                                 u/D
lo           127.0.0.1/8                       u/u
             ::1/128
pppoe0       some.thing.v4                     u/u
             some:thing:v6/64
Funny thing is, pinging from default interface works:
Code:
SSH@10G#ping ipv6 some:thing:else:3901::1
Sending 1, 16-byte ICMPv6 Echo to some:thing:else:3901::1
timeout 5000 msec, Hop Limit 64
Type Control-c to abort
Reply from some:thing:else:3901::1: bytes=16 time<1ms Hop Limit=64
Success rate is 100 percent (1/1), round-trip min/avg/max=0/0/0 ms.
SSH@10G#
But not from the ve 10:
Code:
SSH@10G#ping ipv6 some:thing:else:3901::1 source some:thing:else:3902::2
Sending 1, 16-byte ICMPv6 Echo to some:thing:else:3901::1
timeout 5000 msec, Hop Limit 64
Type Control-c to abort
Request timed out.
No reply from remote host.
And that's with the static route configured:
Code:
ipv6 unicast-routing
ipv6 route ::/0 some:thing:else:3901::1
There is something missing but I can't wrap around my head on what.
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
There is something missing but I can't wrap around my head on what.
SLAAC and DHCPv6 are two different beasts...
But for your last sentence, you are missing maybe a route on the edgerouter for the reply to [some:thing:else:3902::2] via [some:thing:else:3901::2]. ;)
 

rootwyrm

Member
Mar 25, 2017
74
93
18
www.rootwyrm.com
SLAAC and DHCPv6 are two different beasts...
But for your last sentence, you are missing maybe a route on the edgerouter for the reply to [some:thing:else:3902::2] via [some:thing:else:3901::2]. ;)
It's easier than that: that's exactly what it is. The UBNT isn't configured for DHCPv6, it's configured for SLAAC. You can't convert SLAAC to DHCPv6, and they do not behave at all alike. End result is that both ICX and UBNT don't have a valid routing table for [some:thing:else:3902::2] because you haven't done NDP.
 

bbqdt

Member
Sep 15, 2019
93
64
18
Is there a way to configure an icx6610 to get all its ipv6 info from slaac (no hard configured addresses) in, for example, vlan 1 and then provide routing and slaac to vlan 2? Vlan 1 is running pfsense doing slaac and has a /61 to do whatever with.

Could it also do something similar with DHCP and ipv4?
 

jzeus

New Member
Jan 22, 2017
19
4
3
I apologize for doing a terrible job explaining in my prior post. thanks for your time. Please recall I'm a noob; and have not set up any VLANs.

but before I buy a 7150 I wish to make sure it will work for me. ReCap: 7250 is at Server Rack and Top-of-Rack Switch.

7150 would be in another room, a home office. Connection is 10GB Fiber from 7250 to 7150. On the 7150 : I wish to plug in 10GB Workstation on VLAN01, plug in a Printer via CAT6 and have that on VLAN02; and long-term plug in a POE Security Camera into 7150 and have that on VLAN03.

Is this possible? (another poster wrote sentence of " All your devices on the 7150 will be in "that" VLAN that is defined on the 7250. " I wasn't sure how to take this sentence. Can the 7250 setup the above 3VLANS and have them working on the 7150?

(note: I know I lack the skill/knowledge to setup secure VLANS, and I'm seeking $ help to setup pfSense router with two 10GB NICs that go into the TOR 7250.

I am sorry I lack the tech skill to ask my question in a professional manner.

Bluntly, right now, I am not really seeking HOW-TO do the above Wish List, but simply can the 7250+7150 combo do this. (so I know it's ok to start shopping for one)


PS: are you have a 7150 : are they as silent reviewers have stated? how hot does the unit get? can you put your hand on the outside of it?
7150 has no fan. It feels warm outside not hot. I recently replaced the broken 7150 power supply with a MeanWell one and a fan because the MeanWell power supply provides fan power.

I have another broken 7150 which constantly prints

" "PoE Fatal: Vmain (48V) fault has forced all ports on slot 0 to lose power.""

I updated the firmware and the message is gone. The switch is fully functional!
 

rootwyrm

Member
Mar 25, 2017
74
93
18
www.rootwyrm.com
Is there a way to configure an icx6610 to get all its ipv6 info from slaac (no hard configured addresses) in, for example, vlan 1 and then provide routing and slaac to vlan 2? Vlan 1 is running pfsense doing slaac and has a /61 to do whatever with.

Could it also do something similar with DHCP and ipv4?
AFAIK, pretty much nobody supports SLAAC in your proposed scenario. Because, frankly, it's an absolutely terrible idea and not how SLAAC is meant to be used. You are trying to stack dynamics which is Absolutely Not Safe Or Sane. Because SLAAC includes an RA. Which is why you cannot create a valid routing table. You'd be making multiple router advertisements, which are multicast.

Support for SLAAC was only added in 08.0.40. Because the 64xx's do not support SLAAC, you need to configure as an SLAAC pass-through (which means the ICX must behave as layer2 only) and configure appropriate raguard for VLAN isolation, and the UBNT must serve all SLAAC, RA, and NDP.
 

bbqdt

Member
Sep 15, 2019
93
64
18
AFAIK, pretty much nobody supports SLAAC in your proposed scenario. Because, frankly, it's an absolutely terrible idea and not how SLAAC is meant to be used. You are trying to stack dynamics which is Absolutely Not Safe Or Sane. Because SLAAC includes an RA. Which is why you cannot create a valid routing table. You'd be making multiple router advertisements, which are multicast.

Support for SLAAC was only added in 08.0.40. Because the 64xx's do not support SLAAC, you need to configure as an SLAAC pass-through (which means the ICX must behave as layer2 only) and configure appropriate raguard for VLAN isolation, and the UBNT must serve all SLAAC, RA, and NDP.
Any way to do it with dhcpv6?

Or am I stuck using pfsense for ipv6 routing in my vlans due to Comcast changing my ipv6 address delegation regularly?
 

kache

New Member
Jun 27, 2020
15
2
3
AFAIK, pretty much nobody supports SLAAC in your proposed scenario. Because, frankly, it's an absolutely terrible idea and not how SLAAC is meant to be used. You are trying to stack dynamics which is Absolutely Not Safe Or Sane. Because SLAAC includes an RA. Which is why you cannot create a valid routing table. You'd be making multiple router advertisements, which are multicast.

Support for SLAAC was only added in 08.0.40. Because the 64xx's do not support SLAAC, you need to configure as an SLAAC pass-through (which means the ICX must behave as layer2 only) and configure appropriate raguard for VLAN isolation, and the UBNT must serve all SLAAC, RA, and NDP.
So to have SLAAC on the ICX 6xxx we'd lose also ipv4 routing on the switch itself?
 

safado

Member
Aug 21, 2020
42
6
8
These switches don't seem to be very picky about transceivers. One of advantage of using 'supported' transceivers, though, would be optical monitoring. I use these cheap $8 Finisar transceivers in my 6610-48P, and they work fine:
Thanks!! I went ahead and grabbed some since they are remarkably cheap--thank you!

Hi,

i‘m using this one from fs.com without any issues in an ICX6450 to connect to my PC which is equipped with an Intel X550.

fs.com - 10G-SFPP-T
Glad to hear it---ordered a few as well. Appreciate the feedback!

I am using this transceiver with an Intel X550-T1 on the other end:
Code:
ICX7450-24 Switch>sh med eth 1/4/2 
Port   1/4/2: Type  : 10GE SR 300m (SFP+)
             Vendor: UBNT               Version: 2  
             Part# : UF-RJ45-10G        Serial#: XXXXXXXXXXXX
Around $50 from reputable sellers.

My original plan was to use a Chelsio SFP+ card with an SR transceiver... unfortunately, the Linux kernel would panic when trying to suspend/resume with the Chelsio cards (including RJ45 versions). In my experience, the Intel cards just work.
Thanks! I'm guessing the 550 you have is an embedded solution--looks like those cards are 175 or so on ebay. The x520's are more reasonable with SFP+ being far less than the RJ45 models. Wondering if there's any substantial difference between x520, x540's and x550's?

Thanks for the feedback.
 

kache

New Member
Jun 27, 2020
15
2
3
Thanks life runner for the summary. Filling in the gap the 7650 is 48 port multigig 1/2.5/5/10 copper with PoH or 48 port SFP+. 2x40/100gbps stacking/uplinks. I’m about to put one into service next week, if you have any questions would be happy to answer.
How bad is the fan noise?
 

Mitsubishi

New Member
Oct 14, 2018
2
1
3
7150 has no fan. It feels warm outside not hot. I recently replaced the broken 7150 power supply with a MeanWell one and a fan because the MeanWell power supply provides fan power.

I have another broken 7150 which constantly prints

" "PoE Fatal: Vmain (48V) fault has forced all ports on slot 0 to lose power.""

I updated the firmware and the message is gone. The switch is fully functional!
May I ask which fan did you use?
 

rootwyrm

Member
Mar 25, 2017
74
93
18
www.rootwyrm.com
So to have SLAAC on the ICX 6xxx we'd lose also ipv4 routing on the switch itself?
Sorry, should have been clearer there; in order to use any sort of complex IPv6 SLAAC on any ICX at or below 08.0.30, you would have to disable all IPv6 routing and policies on the switch and configure only raguard statements. Should not need to do anything with IPv4. However, if you have any sort of inter-VLAN routing on the switch? It's an automatic non-starter; you will have to trombone all traffic to the prefix's RA'd router and have it route to the correct VLAN. (In other words: don't do that.)
This does not apply for anything running 08.0.40 or later. (It's not the hardware, it's the software. If you were to run 08.0.40 on an ICX6430 then none of this would apply and you'd go to the 08.0.40 guides.)

Any way to do it with dhcpv6?

Or am I stuck using pfsense for ipv6 routing in my vlans due to Comcast changing my ipv6 address delegation regularly?
I don't know UBNT's setup, but yes, DHCPv6 relaying is fully supported on 08.0.30. So if you can make the EdgeRouter do it, then it's doable. (See the reference guide, because DHCPv6 and DHCPv4 are not equivalent.) As for configuring the ICX to get an IPv6 address via DHCPv6, no, that is not supported; static only. (See pp.158 in the L3 guide.) I'd say you'd be much better off configuring the ICX as an IPv6 BGP peer (which is supported) but coping with a frequently changing prefix is not something frrEdgeOS will handle well at all.

However, see above re; Inter-VLAN traffic. With DHCPv6 this does not apply the same. Instead you will need to configure the system for L3 routing and as an L3 router (which is fine, because we are no longer dealing with RA and DHCPv6 can specify router.) Which again, is totally fine and supported. The problem is those changing prefixes. I just do not have an answer for that. But configuring both DHCPv6 relaying and IPv6 routing on 08.0.30 is 100% fine.

Honestly I kind of wonder if this isn't why more 6xxx pulls are popping up and prices are falling - providers realizing they need to get onboard with IPv6 only to find out that DHCPv6 isn't actually a thing (it really is not - it's not a standard, it's not guaranteed, it is an "oh crap we need some duct tape" solution.) So in order to do IPv6 without losing their own minds, they need to bring in SLAAC and router advertisements in a vendor supported package, which means 08.0.40 or above.
 

kache

New Member
Jun 27, 2020
15
2
3
Sorry, should have been clearer there; in order to use any sort of complex IPv6 SLAAC on any ICX at or below 08.0.30, you would have to disable all IPv6 routing and policies on the switch and configure only raguard statements. Should not need to do anything with IPv4. However, if you have any sort of inter-VLAN routing on the switch? It's an automatic non-starter; you will have to trombone all traffic to the prefix's RA'd router and have it route to the correct VLAN. (In other words: don't do that.)
This does not apply for anything running 08.0.40 or later. (It's not the hardware, it's the software. If you were to run 08.0.40 on an ICX6430 then none of this would apply and you'd go to the 08.0.40 guides.)



I don't know UBNT's setup, but yes, DHCPv6 relaying is fully supported on 08.0.30. So if you can make the EdgeRouter do it, then it's doable. (See the reference guide, because DHCPv6 and DHCPv4 are not equivalent.) As for configuring the ICX to get an IPv6 address via DHCPv6, no, that is not supported; static only. (See pp.158 in the L3 guide.) I'd say you'd be much better off configuring the ICX as an IPv6 BGP peer (which is supported) but coping with a frequently changing prefix is not something frrEdgeOS will handle well at all.

However, see above re; Inter-VLAN traffic. With DHCPv6 this does not apply the same. Instead you will need to configure the system for L3 routing and as an L3 router (which is fine, because we are no longer dealing with RA and DHCPv6 can specify router.) Which again, is totally fine and supported. The problem is those changing prefixes. I just do not have an answer for that. But configuring both DHCPv6 relaying and IPv6 routing on 08.0.30 is 100% fine.

Honestly I kind of wonder if this isn't why more 6xxx pulls are popping up and prices are falling - providers realizing they need to get onboard with IPv6 only to find out that DHCPv6 isn't actually a thing (it really is not - it's not a standard, it's not guaranteed, it is an "oh crap we need some duct tape" solution.) So in order to do IPv6 without losing their own minds, they need to bring in SLAAC and router advertisements in a vendor supported package, which means 08.0.40 or above.
Thank you for the explanation!

Perhaps a stupid question: why is it so late and so overcomplicated in the Enterprise space?

My 70€ 2015 Archer had SLAAC fully working in less than a minute, my 2017 500€ Edgerouter Pro took me a week of hitting my head against the wall but eventually I got it working.

How is it possible that a 5000€+ router from 2013 is completely unable to do it?
 

rootwyrm

Member
Mar 25, 2017
74
93
18
www.rootwyrm.com
Thank you for the explanation!

Perhaps a stupid question: why is it so late and so overcomplicated in the Enterprise space?

My 70€ 2015 Archer had SLAAC fully working in less than a minute, my 2017 500€ Edgerouter Pro took me a week of hitting my head against the wall but eventually I got it working.

How is it possible that a 5000€+ router from 2013 is completely unable to do it?
It's not a stupid question at all; it's an issue that people very much do not have half the history on.
Now, I get to say that, because I go all the way back to KAME which remains the reference IPv6 implementation. Well, KAME finished up around 2006. So you'd think "oh jeez, that's even worse." Except KAME's the stack - not the protocols. The RFC for SLAAC wasn't even published in draft until 2007. Emphasis on draft; SLAAC in fact was not formally adopted as a standard until 2015. You want to cite RFC7527. So of course enterprise gear that adheres to formally adopted standards (and actually bothers to certify) didn't support it 3 years before it became a standard! The first drafts of SLAAC date to 2007 and were still waiting on important things like duplicate address detection, router advertisement protocol, and so on.
I've tested Archer's implementation. Unless you updated to a much later firmware, it's actually non-compliant and they can't pass an actual IPv6 certification test. Or even come close. Because it was literally a rushed implementation based on a draft RFC that contained errata compared to the adopted RFC. But it was 'close enough' for home users to not know the difference.

And if you go back to the preceding drafts, it was NEVER envisioned at the time that a routing switch would or should function as a SLAAC. Instead, it would be routers and CMTSes. Routing switches taking over L3 wasn't something anyone really clearly foresaw in 2007. Juniper, who was the class-leader in IPv6 by a wide margin (using the KAME stack from 4.11 with tweaks) at that time, wouldn't offer the EX4200 for another year. They didn't offer DS-Lite until 2010. Cisco didn't have a single-plane L3 switch at all for several more years. And so on.
People act like IPv6 has been around for years, and yes it has. But IPv4 predated DHCP by 17 years. DHCP wasn't actually a standard until 1997. This is the kind of apocrypha you don't know off the cuff - or at all - unless you've been doing this stuff a very, very, very, very long time. And so it is with IPv6. DHCPv6 was thrown together because SLAAC wasn't ready. And then it dragged on and on and on and on.

However, that gets back to "it's software, not hardware." SLAAC and radvd are strictly protocol and software implementations on top of the stack. Adding support for them does not require changing silicon in other words. This does not mean it is just 'plug in software and go.' You have to refactor your overall design to account for these things (particularly things like BGP and OSPF.) Given the ICX6000's were released in 2012 and had a planned EOL in 2018, it was almost certainly decided that the work to put 08.0.40 (released in 2018) on systems originally based on a 2.6.32 kernel (2009, so already 3 years old at time of release) requiring a similarly antique gcc (4.4.7, 2012) for hardware that no longer fit customer profiles (not enough uplinks) was not a sound fiscal decision. Especially since most of this gear is replaced on a 3-5 year cycle anyways. So by the time things like SLAAC started mattering to customers (2018 onward) the 6000's were already on their way out to be replaced by 7000's.

Generally speaking, if you want fully working - much less cutting-edge - IPv6 support? Something made in the last 3 years isn't just recommended, it's pretty much mandatory with most vendors.
 

kache

New Member
Jun 27, 2020
15
2
3
It's not a stupid question at all; it's an issue that people very much do not have half the history on.
Now, I get to say that, because I go all the way back to KAME which remains the reference IPv6 implementation. Well, KAME finished up around 2006. So you'd think "oh jeez, that's even worse." Except KAME's the stack - not the protocols. The RFC for SLAAC wasn't even published in draft until 2007. Emphasis on draft; SLAAC in fact was not formally adopted as a standard until 2015. You want to cite RFC7527. So of course enterprise gear that adheres to formally adopted standards (and actually bothers to certify) didn't support it 3 years before it became a standard! The first drafts of SLAAC date to 2007 and were still waiting on important things like duplicate address detection, router advertisement protocol, and so on.
I've tested Archer's implementation. Unless you updated to a much later firmware, it's actually non-compliant and they can't pass an actual IPv6 certification test. Or even come close. Because it was literally a rushed implementation based on a draft RFC that contained errata compared to the adopted RFC. But it was 'close enough' for home users to not know the difference.

And if you go back to the preceding drafts, it was NEVER envisioned at the time that a routing switch would or should function as a SLAAC. Instead, it would be routers and CMTSes. Routing switches taking over L3 wasn't something anyone really clearly foresaw in 2007. Juniper, who was the class-leader in IPv6 by a wide margin (using the KAME stack from 4.11 with tweaks) at that time, wouldn't offer the EX4200 for another year. They didn't offer DS-Lite until 2010. Cisco didn't have a single-plane L3 switch at all for several more years. And so on.
People act like IPv6 has been around for years, and yes it has. But IPv4 predated DHCP by 17 years. DHCP wasn't actually a standard until 1997. This is the kind of apocrypha you don't know off the cuff - or at all - unless you've been doing this stuff a very, very, very, very long time. And so it is with IPv6. DHCPv6 was thrown together because SLAAC wasn't ready. And then it dragged on and on and on and on.

However, that gets back to "it's software, not hardware." SLAAC and radvd are strictly protocol and software implementations on top of the stack. Adding support for them does not require changing silicon in other words. This does not mean it is just 'plug in software and go.' You have to refactor your overall design to account for these things (particularly things like BGP and OSPF.) Given the ICX6000's were released in 2012 and had a planned EOL in 2018, it was almost certainly decided that the work to put 08.0.40 (released in 2018) on systems originally based on a 2.6.32 kernel (2009, so already 3 years old at time of release) requiring a similarly antique gcc (4.4.7, 2012) for hardware that no longer fit customer profiles (not enough uplinks) was not a sound fiscal decision. Especially since most of this gear is replaced on a 3-5 year cycle anyways. So by the time things like SLAAC started mattering to customers (2018 onward) the 6000's were already on their way out to be replaced by 7000's.

Generally speaking, if you want fully working - much less cutting-edge - IPv6 support? Something made in the last 3 years isn't just recommended, it's pretty much mandatory with most vendors.
Thank you!

So you'd recommend getting a icx 7xxx model if we want to have ipv6 working as intended?
The only one I can get for an acceptable price is the 7450, but from previous posts it appears to be even more noisy than the 6610.
 

rootwyrm

Member
Mar 25, 2017
74
93
18
www.rootwyrm.com
Thank you!

So you'd recommend getting a icx 7xxx model if we want to have ipv6 working as intended?
The only one I can get for an acceptable price is the 7450, but from previous posts it appears to be even more noisy than the 6610.
Pretty much you are going to have to get an ICX 7000 series if you want to do any sort of SLAAC or dynamic addressing on it. The 6000's can, as I mentioned, do IPv6 BGP and OSPF fine. They can do static routing fine. They can even do router advertisements fine (use the raguard statement, etc.) But that's all they can do because that's what we had.

There's some low-cost 7250's out there, but they're not from major resellers. I wouldn't expect to see prices on the 7000's to drop quickly, because they're current supportable models.
 

Fallen Kell

Member
Mar 10, 2020
57
23
8
So I can't seem to find this in the thread (but it probably is), I have been trying and failing to configure a port that uses a default VLAN (even just vlan 1) but can also access other VLANs via tagging.

The moment I add a tagged VLAN to the port I loose access to the device connected to the port on the default vlan, even though the untagged vlan is still configured to vlan 1.

The use case is attempting to connect to a wifi router/access point which has multiple VAPs and VLANs configured on it and accessing it over a SFP+ interface. The SFP+ works fine when I just have the port set as untagged (but I can not reach the other VLANs obviously). Yes it could be an issue with that device, but I should still at least have access to the default VLAN after adding the other tagged VLANs using the untagged VLAN.

My switch is a 6610.
 
Last edited:
  • Like
Reactions: caiparinha