Anything better bang for buck then Microtik CRS317-1G-16S+RM ?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

spyrule

Active Member
Hello,

So I'm looking for a 12+ port SFP+ 10G Switch (Basic switching, with vlan tagging). Preferrably not too loud. Not worried about power consumption or heat.

I'm currently looking at the Microtik CRS317-1G-16S+RM .

It is going to run as a 10Gig switch between 6 total ESXi boxes (3 prod, 3 dev all using vlan tagging to seperate certain traffic) only 1 will be moderately used, plus a Synology SA3400 (dual 10Gig, will be setup in LACP) as the storage location for all the VM's.

So I'd rather get as much 10G throughput as possible. I do not plan on doing any routing, bridging or any other complex switching requirements.

Anybody have an opinion on this device and possibly offer alternative solutions. Looking to keep the price down below 1k CDN (~$800 US)
 
Last edited:

ectoplasmosis

Active Member
Jul 28, 2021
117
53
28
The CRS317 is a bonkers bargain new, especially with recent RouterOS 7.x beta firmware that enables L3 hardware offload for many features including inter-VLAN routing.

I happily run several units in production like this.
 
  • Like
Reactions: Patrick and newabc

NateS

Active Member
Apr 19, 2021
159
91
28
Sacramento, CA, US
Yes, sorry, I dont want a screamer. It is going into an IT closet in the middle of an office, so not too loud if possible. Power is of no concern though. Original post updated.
I wouldn't call the ICX 6610 a screamer -- it's certainly not silent, but it's not extremely loud either. I ran one on my desk for like a week, and while it was slightly annoying, I found it tolerable. A room away and I couldn't even hear it over the fans of my desktop. Unless your network closet is quite poorly isolated and right next to someone's desk, I doubt it would be a problem. I agree with Dave that it would likely be a great option if you're able to buy used.

There's a massive thread about those here: https://forums.servethehome.com/ind...s-cheap-powerful-10gbe-40gbe-switching.21107/
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
I don't know if ICX6610 is somewhat quieter with dual psus/fans and without POE, but my ICX6610-24P (one with POE+) with a single PSU/fan module is fully living up to its 61dBa spec (very quiet office room is around 30dBa)
that said ICX6610 will provide 8 SFP+ ports out of the box and 8 more with qsfp28->sfp+ splitter adapter (total of 16 10gig ports) and all the features and performance you could ever want.
 
  • Like
Reactions: itronin

newabc

Active Member
Jan 20, 2019
465
243
43
The CRS317 is a bonkers bargain new, especially with recent RouterOS 7.x beta firmware that enables L3 hardware offload for many features including inter-VLAN routing.

I happily run several units in production like this.
It is a pretty interesting feature that the Mikrotik is bringing in.

@Patrick , maybe this is a good topic since the Layer 3 hardware offloading will improve the Mikrotik switches' values a lot.
This is the link to the Mikrotik L3 HW offload document.
 
  • Like
Reactions: NateS

itronin

Well-Known Member
Nov 24, 2018
1,234
794
113
Denver, Colorado
I don't know if ICX6610 is somewhat quieter with dual psus/fans and without POE, but my ICX6610-24P (one with POE+) with a single PSU/fan module is fully living up to its 61dBa spec (very quiet office room is around 30dBa)
this is talked about in the Bro-thread - in bursts but scattered. I will add my 2 cents on this since here as well since I have messed with how these switches affect *me*.

To my ears...

2 PSU's and 2 FAN's are actually quieter overall than a single PSU and FAN. There is also less fan ramping.
2 x B psu's are quieter than 2 x A psus.
can't tell difference between B and C.

you can measure db's all you want for loudness but its also about freq distribution and how that effect's each person's physiology (differently).
I run 2 x 6610 48P, dual B's, dual fans, in my basement on a mostly/relatively idle rack. I can't notice them even with the basement door open unless I stand right next to the open door. I can be in the basement room next to them and "notice" them but its not annoying until after more than a few hours of constant close proximity.

fwiw I think my LB6M is louder than a single 6610. which is why its off and I still have it only for emergencies.

again sometimes I think the loudness isn't so much of the problem but more the different freq's (and their relative volume/loundess) and how that affects people differently and negatively (realize age, gender, acoustic history all come into play)
 
  • Like
Reactions: tjk and NateS

NateS

Active Member
Apr 19, 2021
159
91
28
Sacramento, CA, US
I don't know if ICX6610 is somewhat quieter with dual psus/fans and without POE, but my ICX6610-24P (one with POE+) with a single PSU/fan module is fully living up to its 61dBa spec (very quiet office room is around 30dBa)
that said ICX6610 will provide 8 SFP+ ports out of the box and 8 more with qsfp28->sfp+ splitter adapter (total of 16 10gig ports) and all the features and performance you could ever want.
Are you making heavy use of POE? I agree it might be too loud in the faster fan mode, but mine has never hit that. I'm only using the 10gb section, and not using the 1g POE ports at all, which seems to keep the fans in low all the time, and is how I was imagining OP would be using it as well. Possibly if you're putting it in a hotter environment, then the fans might have to speed up even with a lighter load, but at 24C room temperature in my experience they're well behaved.

But yes, beyond that, there are some other things you can do to cut the noise (PSU versions, etc), as itronin mentions, that are detailed in that thread. FWIW my switch with a single rev A power supply and fan I still consider quiet enough for a network closet.
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
Are you making heavy use of POE? I agree it might be too loud in the faster fan mode, but mine has never hit that. I'm only using the 10gb section, and not using the 1g POE ports at all, which seems to keep the fans in low all the time, and is how I was imagining OP would be using it as well. Possibly if you're putting it in a hotter environment, then the fans might have to speed up even with a lighter load, but at 24C room temperature in my experience they're well behaved.

But yes, beyond that, there are some other things you can do to cut the noise (PSU versions, etc), as itronin mentions, that are detailed in that thread. FWIW my switch with a single rev A power supply and fan I still consider quiet enough for a network closet.
Not heavy use (one 1-2 ports), the fans are spitting full blast regardless of PoE load. Like I said and itonin agreed, there are different revisions and dual PSUs/dual fans modules should sound not as loud. Since the switch is my basement it's never bothered me anyhow :)
YMMV
 

nickf1227

Active Member
Sep 23, 2015
197
128
43
33
Not to be a stickler here, but I don't like the design choice you are making at all.
Running beta firmware on a bargain-basement switch brand like Mikrotik in a production environment is just asking for problems. I like Mikrotik. I've owned their switches and put them into production for some other folks. But I would never trust them with a datacenter environment.

If you have 6 servers and a SAN, you need an enterprise-grade solution.

Others have recommended Brocade/Ruckus switches. I also like Brocade/Rockus switches. I have some that have been in production for a long time and have never missed a beat, with 1,200 days of uptime or more. However, I have also had alot of them give me a lot of grief. Most of that grief is with the power delivery modules, so that does not really apply here.

For your specific application, I wouldn't trust any single switch to do the job. You really should be looking at two switches either stacked together traditionally, or with a VSS, vPC, VCS, etc. Two ICX 6610s would be perfect for what you are doing, but they also are EOL and stuck on a firmware that hasn't been maintained in years. If you are fine with that and accept the security implications of that, they'll do the job just fine.

Another option is the VDX 6740, which I have been trumpeting on the forums recently. They are going for $200ish on ebay. Two of those would absolutely do the job. However, getting licensing may be an issue in a production environment.
brocade vdx 6740 | eBay

Another option is going with a Cisco Nexus switch, like the C3064PQ-10GX but then you are looking close to 2,000 dollars not 1,000 to get two of them. No one ever got fired for buying Cisco.
cisco nexus 3064 | eBay


Just my two-cents.
 
Last edited:

ectoplasmosis

Active Member
Jul 28, 2021
117
53
28
Not to be a stickler here, but I don't like the design choice you are making at all.
Running beta firmware on a bargain-basement switch brand like Mikrotik in a production environment is just asking for problems. I like Mikrotik. I've owned their switches and put them into production for some other folks. But I would never trust them with a datacenter environment.
I’d ordinarily agree with you, but the CRS3xx line are such incredible value since the L3-offload became enabled that I was willing to take the risk.

Turns out that for our simple use case involving ~10 VLANs, inter-VLAN routing, IGMPv3 snooping and a mix of storage servers via DACs and clients pulling video via SR optics, the CRS317 has been rock solid for over a year now. Not a single issue or reboot.

It’s also tiny and completely silent, the fans have never spun up.
 
  • Like
Reactions: spyrule

NateS

Active Member
Apr 19, 2021
159
91
28
Sacramento, CA, US
Not heavy use (one 1-2 ports), the fans are spitting full blast regardless of PoE load. Like I said and itonin agreed, there are different revisions and dual PSUs/dual fans modules should sound not as loud. Since the switch is my basement it's never bothered me anyhow :)
YMMV
Even with the worst setup though (single rev A power supply + single fan), it shouldn't need to run full blast at all times without a heavy load or really high ambient temps. My rev A is a bit louder than my rev B, but both switches drop to the low fan speed and stay there after the initial 5 seconds of high when you power it on. You might want to check that your heatsink hasn't fallen off the chip or anything -- a few people in the megathread had that issue due to the plastic retaining pins failing with age and heat.
 
  • Like
Reactions: itronin

spyrule

Active Member
Not to be a stickler here, but I don't like the design choice you are making at all.
Running beta firmware on a bargain-basement switch brand like Mikrotik in a production environment is just asking for problems. I like Mikrotik. I've owned their switches and put them into production for some other folks. But I would never trust them with a datacenter environment.

If you have 6 servers and a SAN, you need an enterprise-grade solution.

Others have recommended Brocade/Ruckus switches. I also like Brocade/Rockus switches. I have some that have been in production for a long time and have never missed a beat, with 1,200 days of uptime or more. However, I have also had alot of them give me a lot of grief. Most of that grief is with the power delivery modules, so that does not really apply here.

For your specific application, I wouldn't trust any single switch to do the job. You really should be looking at two switches either stacked together traditionally, or with a VSS, vPC, VCS, etc. Two ICX 6610s would be perfect for what you are doing, but they also are EOL and stuck on a firmware that hasn't been maintained in years. If you are fine with that and accept the security implications of that, they'll do the job just fine.

Another option is the VDX 6740, which I have been trumpeting on the forums recently. They are going for $200ish on ebay. Two of those would absolutely do the job. However, getting licensing may be an issue in a production environment.
brocade vdx 6740 | eBay

Another option is going with a Cisco Nexus switch, like the C3064PQ-10GX but then you are looking close to 2,000 dollars not 1,000 to get two of them. No one ever got fired for buying Cisco.
cisco nexus 3064 | eBay


Just my two-cents.
First: I have zero need for additional 1Gb ports. Only SFP+ 10Gig.

So to be clear, I really dont plan on doing any routing between vlans or anything else on this device. All routing will be performed by my router (Fortigate 201E). This includes any inter-vlan routing, etc. I dont need IGMP, or anything else, other then perhaps jumbo frames at some point (which, from what I understand, it supports in switch mode anyway).

I would consider the brocades for sure, but literally my only concern is noise with these (I used to use a few LB4M's and those would be too loud... so I'm assuming these are similar in noise level). If I did go with them though, I'd def. do 2 of these switches just to allow for better resiliency more then anything.

Luckily, I wont lose my job over this purchase, but I do have a budget I'm trying to stick too. That's my primary limitation in this instance.

I should also note: This will only be used in production type config for ~2 years or so. After which, all of my publicly accessible VM's will be mothballed (we are in the early stages now of moving to a SaaS cloud service).
 

NateS

Active Member
Apr 19, 2021
159
91
28
Sacramento, CA, US
One thing to consider is do you actually need much bandwidth between your production and dev systems? If not, you might consider putting them on separate stacked switches. There's a lot more options for silent, low-power 10G switches with 8 or less ports. For example, 2x ICX 7250s, which are basically silent, linked by 2x 10G ports gets you your needed 12 SFP+ ports, with the only limitation that you only have 20G between the two halves. Your storage LACP can have one connection to each chassis. If you want redundancy for your production group you could add a third switch to the stack.
 

nickf1227

Active Member
Sep 23, 2015
197
128
43
33
First: I have zero need for additional 1Gb ports. Only SFP+ 10Gig.
That is totally fair, which is why I also mentioned the VDX and the Nexus. They are literally designed for the expressed purpose you are describing. The ICX 6610 is really designed to be a do-it-all type of switch, it's a campus switch with special sauce that would make a great "collapsed core" type of deployment. But that is not what you are asking for, so I understand the concern.

So to be clear, I really dont plan on doing any routing between vlans or anything else on this device. All routing will be performed by my router (Fortigate 201E). This includes any inter-vlan routing, etc. I dont need IGMP, or anything else, other then perhaps jumbo frames at some point (which, from what I understand, it supports in switch mode anyway).
I was not recommending that you change the topology of your network simply because those switches offer additional features over the Mikrotik. I was recommending that you consider other options because it is my opinion that a Mikrotik switch should not be used as the single point of failure for critical server infrastructure.

Consider this for a moment. The MikroTik CRS317-1G-16S+RM weighs approximately 2.5kg, or 5.5 pounds. It consumes a maximum of 44 W of power and has a heatsink sitting in the bakc of the unit with a heatpipe. Where is the airflow to cool the switch coming from? The internal fans are merely recirculating heat inside of the chassis?
1632358973762.png
1632358979725.png

1632358291609.png

Notice how simple that PCB is. There's very few components those fans are cooling... The design literally does not make sense in my opinion. It's 100% form over function. People see the heatsink and think oh look how cool that is!
Now, lets look at the inside of a Brocade ICX 6450, which is the baby brother of the 6610(pic curtesy of @fohdeesha). Notice, it has an internal fan and heatsinks that actually receive airflow over them!
1632358961319.png

This is an older 1Gb switch with a pair of SFP+ ports. It's rated at 65 watts for the non-poe version
1632359235866.png

It's both larger and Heavier than the Mikrotik.
1632359277227.png

There's an old addage that I like to respect. In PC power supplies, heavier is generally better. More heatsinking, higher quality components, bigger capacitors, that sort of thing usually follow. In switches, I feel the same rule applies. I get that we are comparing two vastly different products here-- but my point stands. Even Brocade's EOL old gigabit switch has higher quality components than the Mikrotik....and that is Brocade! Brocade is like the cheap alternative to Cisco, HP/Aruba, Arista, etc, It's not even high end by comparison.

For the sake of the argument, this is the inside of a Cisco Nexus:
1632359660193.png

Let me also quote @fohdeesha from the Brocade thread:
well, it's not particularly fair to compare them, one is an enterprise switch that's $3000 new, and one is a mikrotik device that's $130 new. The Brocade was intended for enterprise and campus environments for the most part and wear items like onboard flash are duplicated for redundancy, the power supply is much higher quality (Delta onboard instead of a wall wart), a lot more ESD input protection on the physical ports/PHY (most of the mikrotik devices I've taken apart have NONE), the software and hardware has been QA'd quite a lot more (I know the ICX engineering & QA team at brocade on its own was larger than all of Mikrotik as a company) - if you want a general idea of Mikrotik's level of QA, read through some of the Mikrotik megathreads here on STH
Lets not forget all of the CVE's active for Mikrotik right now:
Mikrotik Routeros : List of security vulnerabilities (cvedetails.com)

Or the Botnets targeting those vulnerabilities in RouterOS:
MikroTik blog - Mēris botnet

I would consider the brocades for sure, but literally my only concern is noise with these (I used to use a few LB4M's and those would be too loud... so I'm assuming these are similar in noise level). If I did go with them though, I'd def. do 2 of these switches just to allow for better resiliency more then anything.
You have 6 servers and a SAN in that closet. Unless they aren't real servers (Supermicro, HP, Dell, Cisco, etc 1U, 2U) I really don't think the noise of the switch will make any appreciable impact to your sound levels. LB4Ms are loud, totally understand the concern, but you are managing a fairly large deployment of other potentially loud and power hungry devices.

Luckily, I wont lose my job over this purchase, but I do have a budget I'm trying to stick too. That's my primary limitation in this instance.

I should also note: This will only be used in production type config for ~2 years or so. After which, all of my publicly accessible VM's will be mothballed (we are in the early stages now of moving to a SaaS cloud service).
Sure, maybe it's not a permanent solution right now. But plans change. What if you have to live with it for 3, or 4 or 6 years before you transition away? Budgets change, business needs change. As someone who has lived through the pain and is trying to save you from it, I wouldn't be so cavalier as you are in regards to the decision you are making. You obviously care, or you wouldn't have asked the question in the first place.

Imagine coming to work one day, and the entire company is at a standstill because the datacenter switch failed. All productivity is lost, and you do not have another switch to put in it's place. When your boss asks how you that happened, is your answer "because it was cheap and quiet" going to be adequate? I understand you are in a budget, and I am trying to offer alternatives of used enterprise-class hardware, rather than new SMB solutions. If this was for your house, I would not be so adamant here. But it's not.

At the end of the day, it's your decision. I am just trying to help.
 

Attachments

Last edited: