Mikrotik S+RJ10 10GBase-T transceiver with ConnectX-2

[nezach]

TL;DR Has anyone tried Mikrotik S+RJ10 10GBase-T/NBase-T transceiver with ConnectX2 or X3 cards?

I have to relocate my workstation and will only have Cat5e available in the new locaton, so I am looking into NBase-T. I have Mikrotik CSS326-24G-2S+RM switch and Mellanox ConnectX2 card in the workstation. I was thinking about getting S+RJ10 for the switch and Intel X550 adapter. But it would be a lot cheaper to get two S+RJ10 insted of X550. Any ideas if this would work?

Also, anyone has 10GBase-T running over UTP Cat5e? My run to the switch is quite short.

 

[i386]

The cx-2 should support 1gbe gbics, not sure about 2.5 or 5gbe.

 

[nezach]

1Gbps is obviously not a problem, since I have cat5e, but I would prefer 10Gbps or at least 5Gbps. I think I will just buy one transceiver and try things out, problem is I don't have 10GBase-T adapter, but I do have Netgear GS728TXP switch with two 10GBase-T ports. Will report my findings.

 

[nezach]

I finally got my hands on one of these transceivers and had a bit of time to play with it. Overall it is a pretty nice product and so far it looks very promising. ConnectX-2 card accepts it just fine, so does Netgear S3300 switch and of course Mikrotik CSS326.

The only other device that I have with 10GBase-T port is Netgear S3300 switch so all the testing was done between Netgear S3300 and workstation with ConnectX-2 card and S+RJ10 transceiver. There is also ESXi box attached to the switch via SR fiber and one of the guests was used for iperf testing. When my workstation is connected to the switch via fiber I get ~9300 Mbits/sec between workstation and guest VM.

Test results:

1. ConnectX-2/S+RJ10 <-> 50 ft Cat6 UTP patch cable <-> Netgear S3300
   
   • 10 Gb/s link was established, iperf reported ~9100 Mbits/sec throughput.
   • Transceiver specs state that it supports 10GBase-T over up to 100ft Cat6 cable, so this is not surprising and 9100 Mbits/sec is a bit lower than what I get over fiber.
2. ConnectX-2/S+RJ10 <-> 3 ft Cat6 UTP patch cable <-> wall jack <-> ~15 ft. Cat5e UTP cable (in wall) <-> patch panel <-> 3 ft Cat6 UTP patch cable <-> Netgear S3300
   • 10 Gb/s link was established, iperf reported ~9100 Mbits/sec throughput.
   • This is exactly what I need this transceiver for. It is very realistic test in home environment and proves that you can use existing Cat5e cable runs for 10GbE networking. This is of course a very short run.
3. ConnectX-2/S+RJ10 <-> 50 ft Cat6 UTP patch cable <-> wall jack <-> ~15 ft. Cat5e UTP cable (in wall) <-> patch panel <-> 3 ft Cat6 UTP patch cable <-> Netgear S3300
   • this is same location as test #2, but I replaced 3ft patch cable with 50 ft cable.
   • 10 Gb/s link was established, iperf reported ~9050 Mbits/sec throughput.
     
   • Throughput is a bit lower, but cable length is ~65 ft, of course mostly Cat6.
4. ConnectX-2/S+RJ10 <-> 50 ft Cat6 UTP patch cable <-> wall jack <-> ~40 ft. Cat5e UTP cable (in wall) <-> patch panel <-> 3 ft Cat6 UTP patch cable <-> Netgear S3300
   • here I am running 50ft patch cable to a different wall jack with longer Cat5e run to the patch panel
   • 10 Gb/s link was established, iperf reported ~9050 Mbits/sec throughput.
     
   • Things are looking really good here, I am quite impressed it is still performing at this level.

I ran iperf for quite a bit over the longer runs and no RX/TX errors were reported on the switch as well as workstation. I also tried running it over very complicated, and not very realistic, ~170ft Cat5e run with multiple patch cables and wall jacks along the way and it did drop down to 1GbE.

So far I am very happy with this product, it will allow me to use existing Cat5e cabling for 10GbE networking. There is no way I would be able to run new cable for less than the cost of the transceiver. It is also cheaper to get ConnectX-2 card and S+RJ10 than used Intel 10GBase-T adapter and it is more than adequate for home workstation use. The only downside that I have found so far is that some cables with "closed" boots are a bit difficult to disconnect. Monoprice flexboot patch cables do not have this problem though.

I will probably get a second transceiver and will test how things work with S+RJ10 on either end of the line.

 

[Evan]

@nezach thanks for the good testing, really interesting for home type use.

 

[nezach]

I got a second S+RJ10 transceiver and things did not work out as expected at all.

I installed one of them in Mikrotik CSS326 switch and the other in ConnectX-2 card in workstation, connected them using 50ft Cat6 UTP cable and they would only run at 5Gbits/s (5Gbase-T), but at least traffic flow was fine in either direction pushing ~4700 Mbits/sec. Their brochure says that 10Gbase-T is supported over up to 30m Cat6 runs.

Then, I connected them with ~15ft Cat5e run, test #2 in the previous post, and traffic would flow fine at 10Gbit/s in one direction, but not in the other. For some reason flow in one direction encounters massive amount of packet re-transmissions, sometimes it manages to push ~2000 Mbits/sec, but other times only 50 Mbits/sec (see attached image). I spend a lot of time troubleshooting it to no avail. I tried downgrading software on the switch, since it had some issues with SFP transfers in previous versions, replaced the switch with Netgear S3300, ran link only between the switches, taking out ConnectX-2 from the equation, played with TCP windows, etc.. As long as S+RJ10 is on both ends of the link traffic is not flowing well in one direction. If S+RJ10 is connected to S3300 RJ45 port, then things are working fine.

Those transceivers are little black boxes, there are no logs, there is no way to force speed or duplex settings, which makes troubleshooting very difficult.

At this point I am out of ideas and if you guys can recommend anything, I will try it out.

Also, since Mikrotik CSS326 has no fan transceiver gets really hot, almost too hot to touch.

 

[nezach]

I am suspecting that one of the transceivers is defective. The issues have gotten worse, it now sometimes pushes only .5 Mbit/s and I can't find any other explanation to the issues I am experiencing. I will RMA it and hope new one performs as expected.

 

[pcmoore]

First, thanks for sharing your experiences with this transceiver including all of the testing with different cable types/lengths; that was exactly the type of information I was hoping to see in a S+RJ10 review! If you can remember, please report back when you get a replacement module. I was very interested in this transceiver when I first saw it, for the same reasons as you, and was a little disappointed to hear about the S+RJ10 to S+RJ10 performance/heat problems.

 

[nezach]

I finally received replacement for the second transceiver that I purchased and the old one was definitively defective. The new transceiver is not exhibiting problems that the old one was and is performing in line with my expectations from the initial testing.

I repeated tests from post #4, but this time testing was done between workstation with ConnectX-2 with S+RJ10 and Mikrotik CSS326-24G-2S+ with S+RJ10 instead of Netgear S3300.

1. ConnectX-2/S+RJ10 <-> 50 ft Cat6 UTP patch cable <-> Mikrotik CSS326/S+RJ10
   • 10 Gb/s link was established, iperf reported ~9200 Mbits/sec throughput.
     
   • basic test, things are looking good; this is where defective transceiver was only working @ 5Gbase-T
2. ConnectX-2/S+RJ10 <-> 3 ft Cat6 UTP patch cable <-> wall jack <-> ~15 ft. Cat5e UTP cable (in wall) <-> patch panel <-> 3 ft Cat6 UTP patch cable <-> Mikrotik CSS326/S+RJ10
   • 10 Gb/s link was established, iperf reported ~9100-9200 Mbits/sec throughput.
     
   • realistic scenario using short run, things are looking good
3. ConnectX-2/S+RJ10 <-> 50 ft Cat6 UTP patch cable <-> wall jack <-> ~15 ft. Cat5e UTP cable (in wall) <-> patch panel <-> 3 ft Cat6 UTP patch cable <-> Mikrotik CSS326/S+RJ10
   • this is same location as test #2, but I replaced 3ft patch cable with 50 ft cable.
     
   • 10 Gb/s link was established, iperf reported ~9100-9200 Mbits/sec throughput.
     
   • it is still maintaining same throughput as in test #2; this looks better than previous test with Netgear S3300
4. ConnectX-2/S+RJ10 <-> 50 ft Cat6 UTP patch cable <-> wall jack <-> ~40 ft. Cat5e UTP cable (in wall) <-> patch panel <-> 3 ft Cat6 UTP patch cable <-> Mikrotik CSS326/S+RJ10
   • here I am running 50ft patch cable to a different wall jack with longer Cat5e run to the patch panel
     
   • 10 Gb/s link was established, iperf reported ~9100-9200 Mbits/sec throughput.
     
   • throughput is still the same as in test #3 and is definitively higher than what I "clocked" using Netgear S3300

At this point I am pretty happy with the performance and above tests should cover most of the home use cases, but I still want to see it working @ 5Gbase-T. For that I am going to use a cable run from test #2 combined with a loop from the patch panel, located in the basement, to the room on the second floor and then back to the patch panel.

• ConnectX-2/S+RJ10 <-> 3 ft Cat6 UTP patch cable <-> wall jack <-> ~15 ft. Cat5e UTP cable (in wall) <-> patch panel <-> 3 ft Cat6 UTP patch cable <-> patch panel <-> ~65 ft. Cat5e UTP cable (in wall) <-> wall jack <-> 1 ft Cat6 UTP patch cable <-> wall jack <-> ~65 ft. Cat5e UTP cable (in wall) <-> patch panel <-> 3 ft Cat6 UTP patch cable <-> Mikrotik CSS326/S+RJ10
  • in-wall cable run lengths are conservative estimates and total run is ~155ft.
  • all in-wall cables in this test are from Monoprice
    
  • it is finally using 5Gbase-T and iperf is reporting ~4600 Mbits/sec throughput
  • 5Gbase-T is supported over up to 100m/328ft Cat5e runs, so results here are not surprising, but it is still nice to see it performing as advertised.
  • I also replaced 3 ft. patch cable with 50 ft. patch cable, but results stayed the same.

Overall, I am still very happy with the performance of this product. I think it is reasonable to expect at least 5Gbase-T performance from S+RJ10 in any home, unless your cable runs are extremely long or cable is very low quality.

Per port price is quite high when compared with fiber, but if you are dealing with existing Cat5e runs, then this might be a cheaper option than running new cables.

Transceiver does not have a temp probe and I don't have IR thermometer to measure its temperature, but it does get quite hot in fanless Mikrotik CSS326 switch, even when it is not actively utilized. Temp was staying reasonable when running transceiver inside an actively cooled workstation case as well as Netgear S3300 switch. Spec sheet does not specify max temperature, so this might not actually be a problem.

 

[K D]

Your tests convinced me to order one to connect my workstation that's in the secknd floor to my unifi switch in the basement. Hope it plays nice with ubiquiti products.

 

[nezach]

Your tests convinced me to order one to connect my workstation that's in the secknd floor to my unifi switch in the basement. Hope it plays nice with ubiquiti products.

It will be interesting to see.

I forgot to mention that even though transceiver works with Netgear S3300 it does experience a weird issue. If transceiver is inserted while switch is running, it will go into a weird soft reboot and after it recovers only SFP ports are operational, all gigabit ports stop forwarding traffic. If I power it down and then power it back up, everything works fine. I kind of suspect a firmware glitch, but who knows.

 

[Jonkheer]

Thank you for your review. I was postponing my order for my little home lab based on your initial (and other) reviews. Your recent feedback convinced me, so I went ahead and ordered one CRS317, one CRS326 and two RJ10 modules. I hope to 10 Gb/s bridge around 25-30m (100ft) of CAT6 including two wall jacks. I will report my findings when I get all my stuff (I'm still waiting on an ebay Intell X520-DA2 with which I would also like to try the RJ10 module).

 

[Jonkheer]

So I got my gear. First test I did had the following path:
Server <-> DAC CRS317 RJ10 <-> 10m CAT6A <-> Patchbox <-> 22m CAT6 <-> Patchbox <-> 10m shielded CAT6 <-> RJ10 in Intel X520
Around 42m of cable resulted in an iperf3 speed of 5Gb/s

When my fiber gear came in, I was able to lay my final path:
Server <-> DAC CRS317 RJ10 <-> 10m CAT6A <-> Patchbox <-> 22m CAT6 <-> Patchbox <-> 1m Cat6 <-> RJ10 <-> CRS326 <-> SFP+ module <-> OS2 fiber <-> SFP+ module <-> Intel X520
You would expect the same or a better result by eliminating 9m of CAT6 cable, but the iperf3 result was 2.5Gb/s

So I'm a bit disappointed. It seams using the RJ10 module in the X520 resulted in better performance (better link was negotiated). But you would expect using Mikrotik gear between the switches would yield the best result... apparently not

I still have some configuring to do, but I doubt that will improve things. Maybe I'll try to shorten the 10m CAT6A cable on the server side, hoping to see that 5Gb/s again (10Gb/s would be a dream come true).

 

[nezach]

Sucks to hear things are not working out as you expected. I know how frustrating that can be.
Any chance you can eliminate part of the path for testing? Connecting X520 right after the 22m Cat6 run would be a good start. Also, what is "Patchbox" in your path? Keystone jack? I would suggest checking all the connections/cable terminations between cable runs, they are more likely to cause issues than actual cable.

 

[pcmoore]

The CRS326 isn't actively cooled, correct? Based on previous comments that the S+Rj10 tends to run hot, and assuming that the X520 is located in a chassis with some type of active cooling, could it be possible that the transceiver is throttling itself in the CRS326 due to an overheating condition?

I wonder what would happen if you redid the CRS326 configuration above, but with a fan pointed at the switch/transceiver.

 

[Jonkheer]

Regarding the 'patchbox'. It's a wall mount CAT6 unit, but it could be a/the troublemaker as I had to twist the cable hard inside the box to make the connections. I ordered some Cat6a keystones to replace it, hoping that makes things better.
As for the heat, the x520 actually had a bad position in my case (bottom slot, RJ10 facing down, very close to the PSU), so I don't think that's the problem. Maybe the x520 is able to supply more power to the module, resulting in a better connection... I don't know.
I will continue testing and will keep my findings up-to-date in this thread.

 

[Jonkheer]

Small update, I was able to get 4.5Gb/s results with iperf3 (with multiple parallel streams) by just tuning my x520 performance settings (basically reducing the Interrupt Moderation Rate to Low, Minimal, or Off).
Strange thing this was not necessary on the exact same system with the RJ10 module. I must say the X520 is in a slot with PCIe 2.0 x4 lanes, while it is an x8 card. I don't think that's a problem as I will only use one of the two interfaces on the card, so x4 should be enough...
Still, I ordered the keystones, maybe that will help me get 10Gb/s

 

[Jonkheer]

An other small update. Rearranging some stuff on one cable end, I was able to minimize the 10m CAT6A to a 1m CAT7 cable, so, the new path:
Server <-> DAC CRS317 RJ10 <-> 1m CAT7 <-> Patchbox <-> 22m CAT6 <-> Patchbox <-> 1m Cat6 <-> RJ10 <-> CRS326 <-> SFP+ module <-> OS2 fiber <-> SFP+ module <-> Intel X520
Now I'm able to get 10Gb/s
But, when I'm running an Iperf3 receiving test (-R):
- 1P => only gets me 4.8Gbits/s
- 3P => 'full' speed with a result of 9.4Gbits/s
I do notice my system getting more unresponsive running even the -P1 test (I noticed typing this message the cursor lags...) The CPU is far from maxed out (Ryzen 1700X).
When using a send test:
- 1P => gets met 5.4Gbits/s
- 2P => 8.3Gbit/s which is the max I can get...

I hope these speeds will remain (and stable), otherwise I still have those keystones left.
So overall I have still some tuning/testing to do, but the RJ10 delivered its 10Gb/s, which makes me happy

 

[i386]

If you're using windows on both ends, can you try NTttcp?
TechNet NTttcp Utility: Profile and Measure Windows Networking Performance

 

[pcmoore]

I just wanted to report success with a S+RJ10 in a Dell X1052. I was able to establish a 10Gb link between the X1052 and a Dell 8132/N4032 over approximately 90ft (28m) of cat6 with two keystones in the middle (wall plates). While I haven't been hitting the link very hard just yet, it has remained stable for the last 24 hours.

As far as temperature goes, using an infrared thermometer, the exposed portion of the transceiver reads somewhere between 80F and 85F with the ambient temperature at 72F. The fans on the X1052 remain at their usual "low" RPM with no indication of higher than normal temperatures.

Quite happy with the S+RJ10 at the moment, I just ordered another this morning for some additional experiments.