nm, I re-followed @fohdeesha 's advanced config post and now it's working.
This one: ICX6xxx Advanced - Fohdeesha Docs
This one: ICX6xxx Advanced - Fohdeesha Docs
Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
At least there are relatively cheap solution available now to solve that problem if you can live with another switch or a transceiver + PoE injector.Yep, the 6610 is old and doesn't have 2.5GbE PoE, only 1GbE.
I actually have one of my boys connected to one of the 10GbE ports through a MoCA adapter that does 2.5GbE, and it works great, so I know the hardware I have is capable.
I would love to get my hands on a ICX7150-48ZP but these go for $1500+ on eBay right now
Got the 6610 a while ago for $130.
One of those cheap Chinese switches with a 10GbE uplink and PoE should give you at least 3-4 PoE capable ports depending on the model. That's probably the most cost effective solution if you have more than one device that needs 2.5GbE with PoE.
Just a quick note:
For anyone looking for something similar to the ICX6610, but more 40G ports (like me): Juniper EX4300-48P might be the right choice for you.
EX4300-48P vs ICX6610-48P:
PRO: EX4300-48P has 2 more true 40G ports
CON: EX4300-48P has 4 less SFP+ ports AND requires a module (switches on ebay mostly come without)
Neutral: EX4300-48P 40G ports are not breakout capable, that means you have 48x1G + 4x10G SFP+ + 4x40G vs ICX6610 with 48x1G + 16x 10G (8x SFP+, 2x in QSFP) + 2x40G
PRO: EX4300-48P Still gets software updates (if you can get hold of them)
For anyone looking for something similar to the ICX6610, but more 40G ports (like me): Juniper EX4300-48P might be the right choice for you.
EX4300-48P vs ICX6610-48P:
PRO: EX4300-48P has 2 more true 40G ports
CON: EX4300-48P has 4 less SFP+ ports AND requires a module (switches on ebay mostly come without)
Neutral: EX4300-48P 40G ports are not breakout capable, that means you have 48x1G + 4x10G SFP+ + 4x40G vs ICX6610 with 48x1G + 16x 10G (8x SFP+, 2x in QSFP) + 2x40G
PRO: EX4300-48P Still gets software updates (if you can get hold of them)
Looks like another good option!
Right now I'd love to find a 4+ 2.5g PoE+ switch with a 10g sfp+ port for around the $100 mark.
I'd like to run fiber to my detached garage and throw up some PoE cams there. 2.5g would be nice so I could throw up an Omada Access Point out there.
...what kind of cameras are you running that need better-than-gigabit speeds each?
Nah the cams don't need it. I just want 2.5g for the access point. I've also got a touch screen wall mounted computer that I use for my transmission and engine diagrams and wouldn't mind 2.5g there as well.
PoE over NBase-T is a newer standard (802.3bt) and devices are expensive. Even a lower end model like the Netgear MS510TXUP (4 x 2.5G PoE + 4 x 10G PoE + 2 x SFP) is $600 ~ $700 new. I have not seen used models for less than $500
If you don't care about 802.3bt and *managed*, you can go for the Marvell or Realtek based cheap chinese devices - i.e. something like
https://www.amazon.com/YuanLey-Port-PoE-IEEE802-3af-2500Mbps/dp/B0C64QJXW1 - this is 4 x 2.5g PoE + 2 x 10G SFP and costs $80. Technically 802.3 af/at doesn't support NBase-T but most devices will work fine. The problem is that most switches < $600 are un-managed, which *may* cause problems if you're dependent on VLANs etc
If you don't care about managed the TEROW (4x2.5Gbe PoE+, 2x10Gbe SFP, $59.49 with current coupon) is the cheapest thing I was able to find.
If you need managed, I don't think it currently gets cheaper than this one from the 2.5g switch roundup (currently $185). You might be able to bodge together something cheaper out of a non-PoE 2.5Gbe switch and a PoE gigabit switch if you can scrounge up good used deals, but it's be a pain in the backside and probably cost a ton of power.
The other option that comes to mind is a managed PoE gigabit switch with dual SFP+ ports, and slotting one of these into it (which per here should work properly, and is actually available unlike virtually everything else discussed in that thread), but I have no idea if such a thing exists (and I suspect if it does it's a 48-port monster).
EDIT: Actually, depending on the length of the fiber run involved, there's a solid chance it'd be cheaper just to buy two little dumb switches (one 2.5G non-PoE, one gigabit PoE+ [which should be dirt cheap]) and lay two strands, assigning the VLANs back on your managed switch in the house.
Or even skip the 2.5Gbe switch entirely and just have the WAP connected directly to your home switch on a long fiber run and a cheap dumb gigabit PoE switch on copper in the same conduit for the cameras. Would have to run your wall-mount touchscreen off the wifi, but this is almost certainly the cheapest option that accomplishes your goals if you want the cameras on a separate VLAN from the wifi.
Last edited:
That Amazon switch is crazy cheap!
I think I'll order that one next week. That will give me 3 ports for cams, and 1 for an Omada 670. One cam in the garage, one cam on the front pointing down my driveway and one cam on the back corner facing backwards. That leaves me with two sfp+ ports, one for uplink to the main switch and a sfp+ to rj45 for my wall PC.
The garage is about 200ft or so from my "MDF" with 50ft or so if that needing to be buried. I figured I'd buy some cheap plastic car hose and bury it a couple of inches in the ground. (I rent and while we plan on staying for a while I'm not interested in doing it correctly, just as cheap as possible )
I have a 1965 Cyclone in the garage and an Alfa Romeo Gulia in the driveway, I haven't had anyone mess with them yet but I'm sure cameras make for some kinda deterrent. Since I'm moving away from Wyze cams I definitely want to replace them with something.
Last edited:
I bought armored pre-terminated SM cable to connect my house and garage, and it was cheap enough. An armored duplex LC-LC SM 9/125 75 meter cable is around $500.
Honestly that's not bad, when we go to close on a house assuming it has a detached garage I'll probably go that route.
Since this is a rental, I'm probably going to use the tip of a shovel to make a line to the garage and burry a $70 amazon fiber cable a couple of inches below the ground. https://www.amazon.com/Uniboot-Outdoor-Armored-Friction-Compatible/dp/B0BBZZ7W4K
It just needs to last about two years. When we leave I'll snip the ends off
why ruin what is likely a perfectly good cable (assuming it is in the first place)? why? why? why?
if I found fiber ran between the shed and house at a new home I bought but then discovered snipped ends I would hunt them down
Unfortunately it's a rental property. If they find a fiber line in the ground there's a good chance they will charge me the cost to dig it up. Per the rental agreement, I'm not supposed to make any changes to the property.
I'll need to either dig it back up before I move or just snip the ends deep in the ground so it won't be found
Management is kinda dumb but I'd rather not take the risk. I already have to repaint the house or they will charge me for the cat scratches to the walls. (I've paid over $3k in pet fees since moving in and they won't even provide me a bucket of paint) They gave me the paint code and told me I'd be billed for any scratches if I didn't fix them. That kinda petty attitude just says that if they find a buried fiber line that clearly wasn't there when I moved in they will fight me over it and it's just not worth the trouble.
I could call and ask to install it, but they are just going to say no that it's against the rental agreement. I paid a contractor to come out and install my nest thermostat (so it was done "professionally") and they are still fighting me on it even though I'm going to pay to have the original one reinstalled when I leave.
I cannot wait to finally close on a house and be done with rental properties. No HOA and no Condo's just let me work on my cars in peace
Last edited:
Alright fellas.. I also hit a wall. Working with a 6450 and i keep getting a TFTP error of file not found when trying to update_primary. I've verified the directory multiple times.
ICX64XX-boot>> printenv
baudrate=9600
ipaddr=192.168.1.6
serverip=192.168.1.19
netmask=255.255.255.0
uboot=ICX64xx/kxz10105.bin
image_name=icx64xx/icx64r08030u.bin
ver=07.4.01T310 (Jun 29 2012 - 11:04:25)
ICX64XX-boot>> update_primary
ethPortNo = 0
Using egiga0 device
TFTP from server 192.168.1.19; our IP address is 192.168.1.6
Download Filename 'icx64xx/icx64r08030u.bin'.
Load address: 0x3000000
Download to address: 0x3000000
Loading: %
TFTP error: 'File not found' (1)
Try again
Removed the 'icx64xx/icx64r08030u.bin' and it updated. Was I supposed to know that? lol
ICX64XX-boot>> printenv
baudrate=9600
ipaddr=192.168.1.6
serverip=192.168.1.19
netmask=255.255.255.0
uboot=ICX64xx/kxz10105.bin
image_name=icx64xx/icx64r08030u.bin
ver=07.4.01T310 (Jun 29 2012 - 11:04:25)
ICX64XX-boot>> update_primary
ethPortNo = 0
Using egiga0 device
TFTP from server 192.168.1.19; our IP address is 192.168.1.6
Download Filename 'icx64xx/icx64r08030u.bin'.
Load address: 0x3000000
Download to address: 0x3000000
Loading: %
TFTP error: 'File not found' (1)
Try again
Removed the '
Last edited:
If I decide that I don't need any "flexibility" to having to terminate the WAN at the switch, then this DHCP/DNS workaround will still work, right?I'll draw a diagram in a bit (don't have my Visio machine handy, it's at the office).
The problem I was trying to solve (in a somewhat elegant manner) was multi-fold:
- pfSense won't do DNS/DHCP for anything that's not an interface in it. But, if we do that with a layer 3 switch, the question becomes, which device is going to be the gateway for the VLAN? The switch or pfSense?
- Preserve line rate performance between VLANs, at the switch.
- Why do we need to have anything more than a single NIC in a firewall/pfSense device *IF* we're using a layer 3 switch?? The L3 switch IS a router, all we need to do is pass traffic back and forth between the firewall, depending on the rules you setup. If you want to setup rules between VLANs, do it at the switch (it IS a router after all), if you want to setup rules between your internal network and the WAN, that's when you go to your firewall.
- How do I do packet analysis/capture, traffic analysis, IDS etc etc on the WAN, without limiting myself to the hardware/software that runs the firewall?
So, in trying to solve these issues, the lightbulb moment actually came from someone here...@Blue)(Fusion - you're the man! (or woman)
Let's take a layer 3 switch and create 3 VLANS on it.
View attachment 10644
1. The WAN VLAN - This is just a VLAN, add untagged ports to it (at least two, so you can mirror your WAN port and do any additional stuff with that traffic). There is no IP address on this VLAN, no SVI, nothing. Make sure you add the TRANSIT port as tagged to this VLAN.
On the pfSense side, create a VLAN that has the matching ID on the same Nic that is used for the TRANSIT pipe. Don't add an interface based on this VLAN, go into interface assignment, and for the WAN, just choose it from the drop down.
View attachment 10641
Nothing special needs to be done beyond this, other than when we're finally done, reboot the firewall, and do release/renew on the WAN (assuming it's DHCP based).
2. The transit VLAN - This is the VLAN that connects the switch to the firewall. In my case, all it has is a single 10g port, dual-mode (the native VLAN is the transit VLAN), but there's more than one way to skin this cat. The IP address for this VLAN on the switch side is 172.16.0.1/30 (in my case, we only need two IP addresses).
On the pfSense side, there is *no* VLAN for this (remember we added an untagged port on the switch side). The IP address on the pfSense side is static IPv4, 172.16.0.2/30. I named this interface TRANSIT in pfSense. It does not require any additional gateway or anything like that.
View attachment 10642
Go to the firewall rules and allow ICMP on this interface.
View attachment 10637
This will allow you to ping the switch IP (172.16.0.1) from pfSense and the interface IP on the pfSense side (172.16.0.2) from the switch console. These two guys are happy as clams at this point.
On your switch add a default route (0.0.0.0/0) to point to pfSense (172.16.0.2). With that the switch will send all unknown traffic on the transit VLAN to pfSense. If you try to do a nslookup or anything like that on the switch console at this point, it won't work, because we haven't set up the rules yet.
3. Your home/whatever VLAN where the (some of the) devices are - Add whatever untagged ports you need on the switch but also add the transit port as tagged. This will ensure that any unknown traffic from this VLAN will be tagged by the time it hits pfSense over the transit pipe.
In my case, on the switch side, I added a few untagged ports to this, added the tagged transit port and gave it an IP of 10.10.10.1/24. That IP address is important...
On the pfSense side, I created a VLAN with the same ID on the same NIC that the transit interface runs on. Then I added an interface based on that VLAN, and gave it a static IP v4 address of 10.10.10.254/24. No gateway or anything needed. But, we're not done yet.
Enable the pfSense DHCP server on this interface, and give it the following settings.
View attachment 10638
The important parts are:
- Make sure the DHCP range does not include .1 or .254, as we're using those IP addresses, but other than that, nothing special.
- The gateway is set to the IP of the switch for this VLAN. That way, when a device connects, gets an IP address from DHCP, they get a gateway that is on the switch itself. If they access anything else on the switch itself, all L3 routing will be at the switch. The switch will only send traffic to the TRANSIT pipe if it does not know how to route it.
But we're not done yet...
Your device gets an IP address, DHCP is working, the DNS server (you'll be getting the x.x.x.254 IP as the DNS server) is ... inaccessible. That's because we haven't set up the firewall rules yet. So... for this interface in pfSense:
View attachment 10639
I added the ICMP rule for testing, it's not required. Essentially allow DNS traffic as tagged traffic over the TRANSIT pipe.
Now if you do a nslookup on the device, it'll work and resolve the IP, but still no Internet... We're not done yet.
View attachment 10640
Allow the interface net traffic on the TRANSIT interface to whatever you choose, right now it's set to anything in my case. When you're setting up the rule in pfSense, the drop down for source will have the "xxxxinterface net" (which is essentially the IP range). And the switch knows how to route this traffic.
Assuming we've done everything right, you should be up and running with Internet access on that device VLAN. From this point on, knock yourself out adding more VLANS and/or firewall rules.
p.s. (I hope I got everything down accurately. This is actually good, as it's documenting it for me as well.
I don't underatand how you got away not creating any static routes for your Home VLAN though. Because the Home VLAN has an interface added in pfsense, it thinks it is directly connected. So any return traffic from the Internet will end up doing layer 2 in pfsense instead of the actual layer 3 routing back to the l3 switch where the layer 2 should be ending up. Outbound packets go from client>switch>pfsense>WAN while inbound packets will pass from WAN>pfsense>client, skipping the switch so this is a case of asymmetric routing.
Am I missing something here?
Last edited:
No you weren't supposed to know or need to do that, the command in the guide works when copy pasted, if you had to remove the leading folder path, then something got moved around when you extracted the zip archive. Also it's ICX64xx/ICX64R08030u.bin, not icx64xx/icx64r08030u.bin - case matters - this is why I always recommend just copy pasting from the guide and not trying to manually type everything