Cheap 10Gb SFP+ enterprise gear - whats out there?

[86turbodsl]

I've been setting up 10gb to replace my existing 1gb stuff. I have a fiber haul to my workshop about 100M away from the main rack.
The rack has most of the servers in it, 2 proxmox nodes and some other stuff. I have a juniper EX4200 as my main switch, with full POE.
The 10gb stuff, i'm just adding now. I'm looking for lower cost used enterprise stuff if possible, don't need a ton of ports, but would prefer optical/sfp+ so i can keep heat down. I'm aware the wired ones consume a lot of power and make a lot of heat.

At the rack, i can go 19" rack mount, but don't need a ton of ports. At the other end of the backhaul, i need something smaller.
i could probably go as few as 2 sfp+ ports, i need to get the fiber to at least 1 10gb node out there for backup. the remainder of the network could be 1gb to support the mancave and wifi, etc. There's some plc's that also need 100mb. trivial stuff.

I have seen the mikrotik 305 and a couple others, but i think used would be fine if there's something older that could work.

For the house rack, there's gotta be some brocade or dell or something older that's EOL enterprise out there. I have a ton of sfp+ transceivers, so that would be the preferred method.

edit: and oh yes, it needs to be managed. I am trying to segregate traffic on vlans. it'd be better if the switches were all managed at least L2 so i can keep traffic segmented.

 

[sic0048]

There are several great threads on older enterprise network switches here. One of the more common one is the Brocade thread. https://forums.servethehome.com/ind...es-cheap-powerful-10gbe-40gbe-switching.21107

But there are others such as this Mellanox thread.

https://forums.servethehome.com/index.php?threads/mellanox-switches-tips-tricks.39394/

I would suggest you look at those threads (and others) to see if any of that hardware would work in your situation.

 

[BoGs]

I have pretty much your setup. I have in my basement currently an ICX6610 which has a SMF 10G backhaul to an external building 100m away which has a 6u small rack 8 poe devices for AP and cameras on seperate vlans. Then I used 40G breakout cables off ICX6610 for servers in the rack in basement and other SFP+ for another backhaul to my work office. The backhauls are MTP 12 strand fiber so I can attach multiple devices to it, looking at it might have been overkill but being able to burry the cable with armor and waterproof limited options unless you can splice which I cannot.

I am currently upgrading to 25/100G for funsies and to learn more L3 setups.

The ICX6610 is great has 16x10G with breakouts but it uses ~140w of power and not loud to me (R7515 beside it is louder). I will be swapping that out for a ICX7250 which will have only 8x10G SFP+ cages but plan to put in a DCS-7060SX-48Y6C arista as my backbone once I figure out my L3 routed and OPNSense routed vlan setup.

I went overboard and you do not need to but I think ICX7250 (-24P is more expensive then the -48P) might fit your bill and leave more room. I personally stick with Brocade since the thread linked above is invaluable and documentation / videos on how to do stuff is super easy. Arista too. Cisco and Juniper are others that you could look at as they have their L3 small business switches.

 

[BoGs]

Forgot to mention MikroTik would work as well, just make sure you read up on how to program them as the learning curve if you want to advanced things is steep and all community driven.

 

[86turbodsl]

Interesting. I had thought to just add a 2nd switch with primarily SFP+ ports and use that as 10G backbone, but maybe it makes more sense to swap out the Juniper EX4200 for a Brocade 7250? I won't say never, but i'm not sure when i would need faster than 10G. I only upgraded to 1G a year ago lol. The move to 10G is really about wanting to add a node in the shop for "offsite" backup. As the shop is 100M away, short of a large tornado or something, i feel it would protect the data pretty well.

 

[86turbodsl]

Or should i just go for the gusto and get a ICX6610? What would the 7250 give me over that?

 

[BoGs]

Or should i just go for the gusto and get a ICX6610? What would the 7250 give me over that?

The ICX6610 will be cheaper (as its older and louder). You can look at the different "beef" switches that @fohdeesha wrote up initially to get a feel of what people do with them https://forums.servethehome.com/ind...s-cheap-powerful-10gbe-40gbe-switching.21107/.

Overall I have been running the ICX6610 for ~5+ years now and it helped me upgrade from 1G consumer switches to POE everywhere, 10G everywhere with MTP to LC cassettes and 1 40G link to my TrueNas R740xd2. Personally I think with NVME drives and storage is going to become faster and faster. I never want my network to be the bottleneck and its a hobby so I am willing to invest in it. Do what you think is best, any switch from Brocade, Juniper, Cisco, Arista will be miles ahead of consumer switches imo and really better then unifi which I am ripping out after a firmware upgrade wiped the switch and had to reset losing my VLAN mappings on ports. Rendering network down from that as all my stuff was connected on VLAN 99 on that switch.

 

[86turbodsl]

That thread is so gigantic. I'm a bit blown away by it. Ha, the Brocade can't be any louder than the Juniper 4200. It's like a jet engine with dual supplies.

 

[BoGs]

If you go to youtube and search for switches you are looking at - most of the time I have found startup videos to show how loud they are and then eventually calm down after booting. Aristas are the best, they scream when the OS starts

 

[Markess]

For the house rack, there's gotta be some brocade or dell or something older that's EOL enterprise out there. I have a ton of sfp+ transceivers, so that would be the preferred method.

For my "remote rack", which is in the garage where noise and heat aren't as much an issue, I recently picked up a HP 5700 Series non-POE switch ( JG894A). When I was searching Ebay, it seemed that for many manufacturer's gear of a similar generation, the 48 port POE versions were more common and cost less than the lower port count non-POE versions, but also seemed to be a lot less well behaved for power draw.

This one has 48x1gb, 4x10gb, and 2x40gb. At this point I haven't done more than test ports and update the firmware. Documentation states that the 40GB ports will take a 4x10GB breakout DAC cable but haven't tested yet.

After a firmware update fan noise went way down. Draws about 75 watts at idle with a handful of devices connected, which I believe is about the same as similar spec Brocade. Mine came with dual 300w removable PSUs.

Got it for $48 shipped on a best offer. The seller has since raised the asking price quite a bit though (but still has best offer available). For the price I thought it was a no-brainer to try. Although if the prices were similar, I'd probably go with a Brocade.

JG894A HPE Flexfabric 1U Switch 5700 48 1G-RJ-45 and 4 SFP+ and 2 QSFP ports | eBay (I'm not affiliated with the seller)

The home rack is in my home office at the end of my desk, so I try to use only relatively quiet gear there. I'm using an 8x10Gb SFP+ switch from XikeStor (rebranded Seeker unit) that I got on Amazon. Found it when it was on sale for around $110. So far so good. Passively cooled, drawing ~13 watts with 5 devices connected. I've got a gigabit switch in there too for IPMI and other slow stuff.

Good luck with the upgrade!

 

[86turbodsl]

For my "remote rack", which is in the garage where noise and heat aren't as much an issue, I recently picked up a HP 5700 Series non-POE switch ( JG894A). When I was searching Ebay, it seemed that for many manufacturer's gear of a similar generation, the 48 port POE versions were more common and cost less than the lower port count non-POE versions, but also seemed to be a lot less well behaved for power draw.

This one has 48x1gb, 4x10gb, and 2x40gb. At this point I haven't done more than test ports and update the firmware. Documentation states that the 40GB ports will take a 4x10GB breakout DAC cable but haven't tested yet.

After a firmware update fan noise went way down. Draws about 75 watts at idle with a handful of devices connected, which I believe is about the same as similar spec Brocade. Mine came with dual 300w removable PSUs.

Got it for $48 shipped on a best offer. The seller has since raised the asking price quite a bit though (but still has best offer available). For the price I thought it was a no-brainer to try. Although if the prices were similar, I'd probably go with a Brocade.

JG894A HPE Flexfabric 1U Switch 5700 48 1G-RJ-45 and 4 SFP+ and 2 QSFP ports | eBay (I'm not affiliated with the seller)

The home rack is in my home office at the end of my desk, so I try to use only relatively quiet gear there. I'm using an 8x10Gb SFP+ switch from XikeStor (rebranded Seeker unit) that I got on Amazon. Found it when it was on sale for around $110. So far so good. Passively cooled, drawing ~13 watts with 5 devices connected. I've got a gigabit switch in there too for IPMI and other slow stuff.

Good luck with the upgrade!

The xikestor is $160 on amazon right now. But that would work perfect for just SFP+ expansion off the juniper switch. So many ways to go here. I don't have a good handle on how much power the juniper is pulling. I'd guess it's at least 120W as it sits. I have a lot of POE devices, phones, cameras, AP, etc.

 

[86turbodsl]

Well, i jumped in with both feet. I got an offer for a ICX6610 for around 80 shipped in my inbox this morning so i took the plunge. I figure the juniper 4200-48p has similar power consumption to the brocade so replacing one with the other should mainly build out the 10gb connections with no real downside for the rack. Now i just need to come up with something for the shop end.