Suggestion for cheap ToR switch 25GbE with some 100GbE ports

sovking

Member
Jun 2, 2011
83
8
8
Hi guys,
until now I considered DELL S5212F and S224F, as ToR switch 25 GbE with some 100 GbE.
I could like also to consider different alternatives: have you any suggestion ?
 

Rand__

Well-Known Member
Mar 6, 2014
6,390
1,617
113
Mellanox SN2410's?

Or go full 100G and run breakout cables as needed...
 

Scott Laird

Active Member
Aug 30, 2014
305
138
43
I have an Edge-Core AS7326-56X running SONiC. The hardware is quite nice, but SONiC is such a moving target that it's hard to really recommend it at the moment. It's probably fine if you're doing "traditional" datacenter things like a BGP-based spine-and-leaf setup, but weird stuff like VLANS with STP doesn't actually work (although it should soonish).

Also, I can't get it to talk to CX4 or CX5 cards at 100 Gbps right now. I mostly blame Mellanox's weird VPI autodetect thing. I'm waiting for fs.com to send me more DAC cables before I really try debugging.
 

AndreiL

New Member
Jun 30, 2019
23
9
3
It's hard to call anything in 25/100GbE cheap as this is the current generation hardware in many data centers, 200/400GbE being state of art.

Mellanox SN2700 is 25/100GbE (all ports) and seems to be "cheaper" than SN2410 on flea-bay, but still 3-5K.
 
Last edited:

Rand__

Well-Known Member
Mar 6, 2014
6,390
1,617
113
Cant remember exactly but the 51xx series had some serious negative to it, I think when I was looking I skipped those in favour of the aforementioned 52xx series
 

Rand__

Well-Known Member
Mar 6, 2014
6,390
1,617
113
Maybe compare them, negative to me must not be applicable to you after all :)
 

Scott Laird

Active Member
Aug 30, 2014
305
138
43
Last edited:

Rand__

Well-Known Member
Mar 6, 2014
6,390
1,617
113
What cable are you using? I think some of mine didnt work either (but worked fine on the S6100), some I got were broken and some were fine (both MLX IB and ETH cables iirc)
 

Scott Laird

Active Member
Aug 30, 2014
305
138
43
I have a couple generic fs.com 100G 2m DAC cables as well as a couple I picked up elsewhere. I haven't done a whole lot of testing or experimentation yet; last night was the first time I actually tried to get 100G working. I'm able to get a link between pairs of ports on the switch with one of the cables, but I haven't tested the others. So there's still a lot of room for random errors and bad hardware.
 

Rand__

Well-Known Member
Mar 6, 2014
6,390
1,617
113
Ah alright, then good luck.
Hope you don't have to go down my rabbit hole :p Finding the FEC issue was a very long and quite expensive process;)
 

Scott Laird

Active Member
Aug 30, 2014
305
138
43
Heh. I *almost* ordered a set of 100G-SR optics and a short MTP trunk fiber yesterday, but sanity prevailed. Even if those worked, it wouldn't really be useful long-term. It's not like I'm going to pay an extra $250/link above the cost of 100G NICs and switches.
 

Rand__

Well-Known Member
Mar 6, 2014
6,390
1,617
113
lol
yeah, been there done that. although paid significantly less for mine thankfully. but all in all... better not think about it.
 

Scott Laird

Active Member
Aug 30, 2014
305
138
43
How did you configure FEC? Is the command above right? Do you need to re-plug cables or anything?

I'm still not getting anything connected, with 2 different cables on 2 ports and an assortment of settings on both ends.
 

Rand__

Well-Known Member
Mar 6, 2014
6,390
1,617
113
I never noted the command so it must have been simple.
I'll try to seach old debug files in the morning