Anything better bang for buck then Microtik CRS317-1G-16S+RM ?

[nickf1227]

The fan on the MikroTik CRS317 is there to remove heat from the SFP+ cages, not for the switch chip.

https://www.servethehome.com/mikrotik-crs317-1g-16s-rm-review/

Maybe. But if you notice, there are no other switch manufacturers in the universe that cool the switch chip with a passive radiator. If that was a good design, wouldn't other have tried it before or copied it after?

It isn't the best switch in the world, but it's low power and can be quiet. Under $800 and you can get two for redundancy. I'm not sure about exchange rates, but MikroTik can get very close to giving you two for that budget.

I'm not bashing the switch. If it were being used as a high performance access switch, I would say that it's a great buy. If you are like a content creator like Linus Tech Tips and you have a bunch of editors who need a high performance connection to a central server for editing, it's a fantastic buy. If you are a home user who wants more performance, its also a fantastic buy. But it's not a datacenter switch.

As for the buying two of them idea. Sure, maybe. But how do you design that architecturally? How are you load balancing your production traffic at L2? Lets say you build a dvswitch in VMWare that has a few different VLANs on it. You need to assign uplinks to those port groups. You can't do a LAG across both switches, so you have a single interface going to each, and in VMWare you fail over from one to the other. But now you are losing capacity. You can assign both uplinks as "Active" and do mac-hash load balancing, and sure that will work and will help increase your capacity. But now you have two links with traffic going across them with no redundancy.

If we use the car analogy, the Mikrotik switch in question is a '95 Civic Hatchback. It's light, it's nimble, and you dropped in a turbocharged H22 motor so it's fast. Maybe even 2 fast
A real datacenter switch is more like a Ford Raptor R or Dodge Ram 1500 TRX. They are work horse trucks, but they also go fast.

Do you see how that's not the same thing? My argument isn't that MikroTik is necessarily bad (although the vulnerabilities in RouterOS do not give me the warm and fuzzies), my argument is that you should buy the right tool for the job. You can open a can of vegetables with a steak knife (ask me how I know!) But can openers exist for a reason.

 

[ljvb]

The biggest concern, is support, you are trying to build something out cheap. With the Mikrotik, you probably won't get great support, nor longevity. With the brocade, of which I have 2, you will get the quality, longevity and the features you want, but you still won't be getting any support.
So in all honesty, neither of the options are really ideal.

It looks like you have a budget, but that cost is only the initial purchase, if you cheap out, you are going to destroy that budget further down the line in the event of a failure, whether it is replacement hardware and/or man hours getting back up and running.

At the very least, get a campus switch with the stats you need from a reputable company with a support contract.. you will regret it otherwise.

 

[nickf1227]

The biggest concern, is support, you are trying to build something out cheap. With the Mikrotik, you probably won't get great support, nor longevity. With the brocade, of which I have 2, you will get the quality, longevity and the features you want, but you still won't be getting any support.
So in all honesty, neither of the options are really ideal.

It looks like you have a budget, but that cost is only the initial purchase, if you cheap out, you are going to destroy that budget further down the line in the event of a failure, whether it is replacement hardware and/or man hours getting back up and running.

At the very least, get a campus switch with the stats you need from a reputable company with a support contract.. you will regret it otherwise.

+111111111
I wasn't even going there with my posts, but you are 1000% correct.
Third-party post-warranty support contracts with reputable companies are invaluable for these type of reasons. I've had some very good experience getting support contracts on old hardware that has eventually failed over the past few years, and the costs are fairly reasonable.

Sparing is also a valid option, having a backup device staged and ready is a perfectly legitimate strategy in a pinch.

 

[ectoplasmosis]

Maybe. But if you notice, there are no other switch manufacturers in the universe that cool the switch chip with a passive radiator. If that was a good design, wouldn't other have tried it before or copied it after?


I'm not bashing the switch. If it were being used as a high performance access switch, I would say that it's a great buy. If you are like a content creator like Linus Tech Tips and you have a bunch of editors who need a high performance connection to a central server for editing, it's a fantastic buy. If you are a home user who wants more performance, its also a fantastic buy. But it's not a datacenter switch.

As for the buying two of them idea. Sure, maybe. But how do you design that architecturally? How are you load balancing your production traffic at L2? Lets say you build a dvswitch in VMWare that has a few different VLANs on it. You need to assign uplinks to those port groups. You can't do a LAG across both switches, so you have a single interface going to each, and in VMWare you fail over from one to the other. But now you are losing capacity. You can assign both uplinks as "Active" and do mac-hash load balancing, and sure that will work and will help increase your capacity. But now you have two links with traffic going across them with no redundancy.

If we use the car analogy, the Mikrotik switch in question is a '95 Civic Hatchback. It's light, it's nimble, and you dropped in a turbocharged H22 motor so it's fast. Maybe even 2 fast
A real datacenter switch is more like a Ford Raptor R or Dodge Ram 1500 TRX. They are work horse trucks, but they also go fast.

Do you see how that's not the same thing? My argument isn't that MikroTik is necessarily bad (although the vulnerabilities in RouterOS do not give me the warm and fuzzies), my argument is that you should buy the right tool for the job. You can open a can of vegetables with a steak knife (ask me how I know!) But can openers exist for a reason.

You really have beef with Mikrotik for some reason!

The CRS3xx series now also have support for MLAG in hardware, so that’s redundancy taken care of.

In my production CRS317 units, the fans hardly ever spin up. The switch chip is tiny and low-power; the rear heat sink is more than enough to cool it passively.

I’d have no qualms in recommending two CRS317 units in MLAG as a cheap & cheerful redundant ‘core’, with a third unit sat as a warm spare.

I run a variety of Cisco and Edge-Core 10/25/100Gb switches at our sites, and the CRS317 units have been no less reliable than this ‘datacentre’ hardware.

As I said in my initial post, they are incredible value.

 

[nickf1227]

You really have beef with Mikrotik for some reason!

The CRS3xx series now also have support for MLAG in hardware, so that’s redundancy taken care of.

In my production CRS317 units, the fans hardly ever spin up. The switch chip is tiny and low-power; the rear heat sink is more than enough to cool it passively.

I’d have no qualms in recommending two CRS317 units in MLAG as a cheap & cheerful redundant ‘core’, with a third unit sat as a warm spare.

I run a variety of Cisco and Edge-Core 10/25/100Gb switches at our sites, and the CRS317 units have been no less reliable than this ‘datacentre’ hardware.

As I said in my initial post, they are incredible value.

You really have beef with Mikrotik for some reason!

The CRS3xx series now also have support for MLAG in hardware, so that’s redundancy taken care of.

In my production CRS317 units, the fans hardly ever spin up. The switch chip is tiny and low-power; the rear heat sink is more than enough to cool it passively.

I’d have no qualms in recommending two CRS317 units in MLAG as a cheap & cheerful redundant ‘core’, with a third unit sat as a warm spare.

I run a variety of Cisco and Edge-Core 10/25/100Gb switches at our sites, and the CRS317 units have been no less reliable than this ‘datacentre’ hardware.

As I said in my initial post, they are incredible value.

First I want to point out that I didn't know MLAG was available on router OS. That's neat so I apologize on that front.

In any case I'm not disputing the value of the Mikrotiks. I don't have a bone to pick with then. They are great switches for what they are. But market segmentation exists for a reason. The more complex PCBs and higher power draw on enterprise and Datacenter class switches exist for a reason too.

If you are running yours in a config like that or wouldn't mind running some in the way you described, power to you. I wouldn't, and I don't think my reasons are irrational. However we may be at an impass so :shrug:

 

[ectoplasmosis]

First I want to point out that I didn't know MLAG was available on router OS. That's neat so I apologize on that front.

In any case I'm not disputing the value of the Mikrotiks. I don't have a bone to pick with then. They are great switches for what they are. But market segmentation exists for a reason. The more complex PCBs and higher power draw on enterprise and Datacenter class switches exist for a reason too.

If you are running yours in a config like that or wouldn't mind running some in the way you described, power to you. I wouldn't, and I don't think my reasons are irrational. However we may be at an impass so :shrug:

I completely agree with you in the sense of ‘right tool for the job’, however...

In the context of this thread, where the user needs something low-budget and readily available, the trade off between trying to get hold of second-hand enterprise switches of unknown provenance via a fluctuating second-hand market with zero warranty or support, potential licensing issues to contend with etc, and buying brand-new ‘SMB’ units that are at least warranted to last a year and are fresh from the factory with all features enabled and lifetime firmware updates and online ticket support included in the price, is something that the original poster will need to consider.

I would be hesitant to go with decommissioned second-hand enterprise gear in their position.

 

[ljvb]

I completely agree with you in the sense of ‘right tool for the job’, however...

In the context of this thread, where the user needs something low-budget and readily available, the trade off between trying to get hold of second-hand enterprise switches of unknown provenance via a fluctuating second-hand market with zero warranty or support, potential licensing issues to contend with etc, and buying brand-new ‘SMB’ units that are at least warranted to last a year and are fresh from the factory with all features enabled and lifetime firmware updates and online ticket support included in the price, is something that the original poster will need to consider.

I would be hesitant to go with decommissioned second-hand enterprise gear in their position.

The OP noted 6 ESXI servers, and a SA3400 array.. that alone is probably $20k if not more, the 3400 is around $8k on it's own without drives.... And then you want to throw a $400 switch in the mix with little to know redundancy, and more than likely not timely business support.. There are 3 prod and 3 dev machines.. I'm sorry, I am all for cheap hardware (hell I remember having to run snort IDS machines on dell workstations in a production environment in a very large federal agency, 50k+ employee sized, in the early 2000s,, because no one sees ROI on security, well back then anyways, we had to scavenge).

But there comes a point, where you have to front the upfront costs to avoid massive costs later at best, or catastrophic failures later causing major business operations interruptions, or at worst, permanently losing production data and environments and potentially going out of business...

 

[ectoplasmosis]

The OP noted 6 ESXI servers, and a SA3400 array.. that alone is probably $20k if not more, the 3400 is around $8k on it's own without drives.... And then you want to throw a $400 switch in the mix with little to know redundancy, and more than likely not timely business support.. There are 3 prod and 3 dev machines.. I'm sorry, I am all for cheap hardware (hell I remember having to run snort IDS machines on dell workstations in a production environment in a very large federal agency, 50k+ employee sized, in the early 2000s,, because no one sees ROI on security, well back then anyways, we had to scavenge).

But there comes a point, where you have to front the upfront costs to avoid massive costs later at best, or catastrophic failures later causing major business operations interruptions, or at worst, permanently losing production data and environments and potentially going out of business...

I completely agree, the OP should be looking to allocate significantly more budget to this.

But given that they asked for something below £1K, I stand by my suggestion that a couple of brand new CRS317 units in a MLAG setup, with their dual PSUs, plus a third ‘warm spare’ would suit the OP better than years-old second-hand enterprise kit of unknown provenance and no warranty/support/guarantee of ongoing license or firmware maintenance.

 

[MiniKnight]

The passive on switch chip part is also because the switch chip being used is a lower power chip. It's like how you could get a Sandy Bridge chip for a server or a Denverton and get similar performance but at very low power

 

[nickf1227]

I completely agree, the OP should be looking to allocate significantly more budget to this.

But given that they asked for something below £1K, I stand by my suggestion that a couple of brand new CRS317 units in a MLAG setup, with their dual PSUs, plus a third ‘warm spare’ would suit the OP better than years-old second-hand enterprise kit of unknown provenance and no warranty/support/guarantee of ongoing license or firmware maintenance.

I too still stand by my recommendation as well. Buying "the right tool" for the job is still a better strategy than settling for less. Couple that with writing an executive risk statement asking for funding for post-warranty support from someone like: Park Place Technologies - Global IT Infrastructure Service Provider or another reputable company and you have successfully CYA. If they give you additional funding for warranty support, great. If not, they are accepting the risk, not you the IT admin. I respect your opinion as well, and at the end of the day the OP will do whatever he thinks is best. I am just trying to advise to the best of my ability from my prior experience.

The passive on switch chip part is also because the switch chip being used is a lower power chip. It's like how you could get a Sandy Bridge chip for a server or a Denverton and get similar performance but at very low power

You are drawing an incorrect comparative. Let me illustrate the differences another way, following this theme.

I have a Cisco UCS C220 M3 as my home production server. It has a E5-2660 V2 which is an Ivy Bridge Xeon.



I also have had several TinyMiniMicro units, several low end single socket Xeon servers like the Dell T140, and other SuperMicro 1U Single socket servers. I used these low end computers to build servers for lab usage in collaboration with @Patrick last year. I've written several articles here on STH discussing these types of topics: Nick Fusco, Author at ServeTheHome.


You can certainly use lower end servers or even 1-liter computers to accomplish similar goals to the ones my UCS is doing. In fact, some of the 1-liter computers may even have substantially faster single core performance than my Xeon.


But they are definitely not the same thing. Patrick does a really good job at explaining this in the video/article: Of BBQ and Virtualization Why Large Nodes Reign - ServeTheHome

The difference between the Mikrotik switch and 5-year old used enterprise or datacenter switches is not dissimilar. They are simply in a different market segment, with different intended purposes. Sure you can use the Mikrotik switch as a datacenter switch. But you also can build a Proxmox Cluster or VMWare vSAN with a bunch of corporate super small formfactor desktops. That doesn't make either of those things the correct way to do it in a business environment.

 

[MiniKnight]

Ok but wow. You're using a V2 10 core and its like 30% faster than a 6C TMM? That's crazy

 

[nickf1227]

Ok but wow. You're using a V2 10 core and its like 30% faster than a 6C TMM? That's crazy

Well there's two...because it's dual socket

 

[spyrule]

Changed my mind. I do like the idea of buying 2-3 enterprise grade switches, and for the cost without a warranty/support contract. Luckily, I have until ~Dec to work out any issues. If they fail, then I'll buy the CRS or Unifi's aggregation switch (tbd).

 

[nickf1227]

Changed my mind. I do like the idea of buying 2-3 enterprise grade switches, and for the cost without a warranty/support contract. Luckily, I have until ~Dec to work out any issues. If they fail, then I'll buy the CRS or Unifi's aggregation switch (tbd).

Let us know how it goes or if you need assistance

 

[spyrule]

Let us know how it goes or if you need assistance

So I ordered the Microtik, changed my mind and cancelled the order, but they had already shipped it. So I guess I'm going to try the Microtik and see how it goes. Worst case, I get performance issues and order the other two switches anyway. Now i just need to figure out with DSM 7 if I should go iscsi or NFS.... tbd.

 

[ectoplasmosis]

So I ordered the Microtik, changed my mind and cancelled the order, but they had already shipped it. So I guess I'm going to try the Microtik and see how it goes. Worst case, I get performance issues and order the other two switches anyway. Now i just need to figure out with DSM 7 if I should go iscsi or NFS.... tbd.

The only time I’ve seen issues with my CRS317s is when pushing a lot of packets via Layer 3 forwarding between 10Gb/s and 1Gb/s clients, which causes the relatively small switch chip buffer to fill up and a small amount of dropped packets.

Enjoy your cheap-as-chips 10Gb switch!