This summarizes my foray into 10GbE networking.
I started out with the MikroTik CRS309-1G-8S+IN SFP+ switch in my basement. I have servers and a NAS all connected via SFP+ DAC working well with the switch. I'm pretty happy with the lower cost and lower power usage of SFP+ based equipment. I'm not a fan of MikroTik's RouterOS, though. IMO, RouterOS is a bit unintuitive, and I don't like how it shows features/functionality that the actual switch is incapable of (such as wireless functionality showing up in RouterOS, despite the switch having no wireless functionality) and the switch providing functionality that it may not be very good at doing since certain features would require high CPU usage. So, when playing with the software, it's not always obvious what software features the hardware does well, doesn't do well, or can't/shouldn't do at all. But other than that, I've been pretty happy with the MikroTik switch's hardware (and more importantly, low cost) and overall performance for what I use it for.
Now that I got bitten by the 10GbE bug, I decided I wanted 10GbE in other parts of my home. All the rooms in my home have Ethernet (well, not the bathrooms), so I wanted to take advantage of existing cabling. 90% of my in-wall cables are CAT 6, and the remaining are CAT 5e that were installed almost 20 years ago. I'm in no mood to do more wiring in my home, so I wanted to try multi-gigabit switches.
Previously, I got the Netgear XS505M (5 -port) multi-gigabit unmanaged switch. But after seeing the ZyXEL XS1930-10, I decided to try it it (compared to the Netgear XS505M, the ZyXEL has double the ports, is not significantly more money for double the ports, and it's a managed switch). So, I ordered a ZyXEL XS1930-10 from Provantage to test mult-gigabit.
I got the MikroTik S+RJ10 SFP+ to 10Gbase-T module so I can connect the MikroTik switch in my basement to the existing CAT 6 cable going up to the central equipment closet in the 2nd floor. In the equipment closet, I patched the basement port to the port for my 1st floor office. I installed a ZyXEL XS1930-10 Multi-gigabit switch in my office. So, connectivity was:
Basement MikroTik <===> 2nd floor equipment closet patch cable <===> 1st floor Office ZyXEL
With this setup, I was only getting 5GbE performance between the switches :-(. I had hoped to get 10GbE, since I estimated I was still below the theoretical limit of what Cat 6 ought to be able to do to get 10GbE performance.
Then, my 2nd ZyXEL XS1930-10 finally arrived that I ordered from zyxelguard (Provantage was out of stock. I called zyxelguard, and I got an even lower price than their online advertised price. Since they are in the West coast and I'm in the East coast, it took a week to receive (you could pay more for 2 day ground). I guess
@PigLover is in the West coast so he got his the next day.
So, when my 2nd ZyXEL arrived, I put it in the 2nd floor equipment closet next to my core Cisco switch.
Hence, I changed my connections to:
Basement MikroTik <===> ZyXEL switch in 2nd floor equipment closet <===> 1st floor Office ZyXEL
So, having the ZyXEL in the equipment closet basically "reduced" the length of the CAT 6 cable run from the basement to my office.
With this setup, I got 10GbE connectivity between the MikroTik basement switch and my ZyXEL office switch! Perfect!
Then, for one of my "edge" switches, I decided to get the Netgear MS510TX "funky switches " (Patrick's words), composed of 4 x1 GbE, 2 x 2.5 GbE, 2 x 5 Gbe, 1 x 10GbE, and 1 x SFP+.
I ran around to most of the rooms in my home with the Netgear switch just to test what connectivity each room can get with the ZyXEL in the main equipment closet. I was pleasantly surprised that, in all the rooms I tested, the Netgear and ZyXEL switches established 10GbE connectivity with each other, even on old CAT 5e cabling from a couple of rooms wired almost 20 years ago (granted, the wiring from these rooms were less than 50'). I haven't run any iPerf testing yet, but my switches all showing 10GbE in their web GUI and LED lights was a good sign.
But what I really wanted to do is to use the Netgear switch in the Sunroom. This "funky" switch would be perfect for my Sunroom, where I wanted SFP+ for a PC, 1GbE for a number of devices such as a printer, and (for now) 1GbE for a WiFi AP (and, in the future, the switch can support 2.5GbE if needed for future WiFi AP [ I don't need POE here, since I can always use a nearby outlet for a WiFi AP. Besides, the POE version of the Netgear switch is much more expensive]).
I really wanted to get > 1GbE performance in my Sunroom. This room has always been problematic with WiFi, since it is an extension to the original part of the house. One of the walls in the Sunroom is the original exterior wall of the house, which is made of stone. Sometimes, WiFi would work well, sometimes it would be terrible. So one day, several years ago, I decided I really wanted Ethernet to the room. Unfortunately, there was no easy way to route an Ethernet cable from the central closet to the room. I had to go outdoors: the first part of the cable from the 2nd floor equipment closet is indoors, but the 2nd half or so of the cable actually runs outside the house before it goes back indoors. At the time, outdoor-rated CAT 6 cable was not readily available (and this was before multi-gigabit was in my radar). So, I used CAT 5e cable. The cable length was about 130 feet. I wasn't too confident on what performance I can get out of a 130' CAT 5e outdoor-rated cable.
I put the Netgear switch in the Sunroom. Lo and behold, with a 130' CAT 5e cable, I got 10GbE connectivity between the Sunroom and my equipment closet in the 2nd floor!! I wonder if the fact that the cable runs outdoors means there is no interference/cross talk from other sources, which help achieve the 10GbE performance.
The ZyXEL and Netgear also have SFP+, and for those, I plan on using Mellanox NICs, like I did for my other basement servers.
So far, I'm pretty pleased with the multi-gigabit switches. Netgear and ZyXEL web GUIs are okay (as Patrick noted, Netgear not having user names and a silly default password of "password" are ridiculous). I still like Cisco GUI the best, but not their multi-gigabit switch availability/prices.
Having said that I'm thinking that I may in the not too distant future replace my old Cisco SG200-26 with maybe the Cisco SG350X-24 switch (It has 4 x SFP+ that may be useful to use as uplink to the ZyXEL). Any thoughts on SG350x-24?
One caveat about the ZyXEL that I noticed. If it's warmer than about 75 degrees F ambient temperature, I noticed its fan is silent for 5 to 10 seconds, and then the fan ramps up for a second or so, then is quiet for another 5 to 10 seconds, then it ramps up for a second or so, then is quiet, and so on and on... I sit about 2 feet from the switch, so I do hear it. In a way, it might be better if it were just running on a very low RPM constantly, rather than ramping up, being quiet, ramping up, etc. My other ZyXEL is in an equipment closet, which is in a guest room. You won't really hear it when 10' away even if the equipment closet door is open.