Aruba MAS series SFP+ & POE+ switches sub-$100

pr1malr8ge

Member
Nov 27, 2017
63
20
8
40
I'm an idiot, and did not know that you needed to add threads to Iperf3 to get higher bitrates.

Code:
c:\iperf>iperf3 -P 8 -c 192.168.254.200
Connecting to host 192.168.254.200, port 5201
[  4] local 192.168.254.1 port 53756 connected to 192.168.254.200 port 5201
[  6] local 192.168.254.1 port 53757 connected to 192.168.254.200 port 5201
[  8] local 192.168.254.1 port 53758 connected to 192.168.254.200 port 5201
[ 10] local 192.168.254.1 port 53759 connected to 192.168.254.200 port 5201
[ 12] local 192.168.254.1 port 53760 connected to 192.168.254.200 port 5201
[ 14] local 192.168.254.1 port 53761 connected to 192.168.254.200 port 5201
[ 16] local 192.168.254.1 port 53762 connected to 192.168.254.200 port 5201
[ 18] local 192.168.254.1 port 53763 connected to 192.168.254.200 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   100 MBytes   842 Mbits/sec
[  6]   0.00-1.00   sec   102 MBytes   851 Mbits/sec
[  8]   0.00-1.00   sec  66.0 MBytes   553 Mbits/sec
[ 10]   0.00-1.00   sec   100 MBytes   840 Mbits/sec
[ 12]   0.00-1.00   sec   101 MBytes   849 Mbits/sec
[ 14]   0.00-1.00   sec   100 MBytes   838 Mbits/sec
[ 16]   0.00-1.00   sec  99.2 MBytes   832 Mbits/sec
[ 18]   0.00-1.00   sec   100 MBytes   843 Mbits/sec
[SUM]   0.00-1.00   sec   769 MBytes  6.45 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   1.00-2.00   sec   110 MBytes   920 Mbits/sec
[  6]   1.00-2.00   sec  73.8 MBytes   619 Mbits/sec
[  8]   1.00-2.00   sec   113 MBytes   950 Mbits/sec
[ 10]   1.00-2.00   sec   109 MBytes   918 Mbits/sec
[ 12]   1.00-2.00   sec   110 MBytes   924 Mbits/sec
[ 14]   1.00-2.00   sec   109 MBytes   911 Mbits/sec
[ 16]   1.00-2.00   sec   108 MBytes   910 Mbits/sec
[ 18]   1.00-2.00   sec   111 MBytes   934 Mbits/sec
[SUM]   1.00-2.00   sec   845 MBytes  7.09 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   2.00-3.00   sec   120 MBytes  1.01 Gbits/sec
[  6]   2.00-3.00   sec   121 MBytes  1.01 Gbits/sec
[  8]   2.00-3.00   sec   120 MBytes  1.00 Gbits/sec
[ 10]   2.00-3.00   sec  76.5 MBytes   641 Mbits/sec
[ 12]   2.00-3.00   sec  86.0 MBytes   720 Mbits/sec
[ 14]   2.00-3.00   sec   118 MBytes   987 Mbits/sec
[ 16]   2.00-3.00   sec   118 MBytes   986 Mbits/sec
[ 18]   2.00-3.00   sec  77.4 MBytes   648 Mbits/sec
[SUM]   2.00-3.00   sec   837 MBytes  7.01 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   3.00-4.00   sec   116 MBytes   971 Mbits/sec
[  6]   3.00-4.00   sec   116 MBytes   970 Mbits/sec
[  8]   3.00-4.00   sec   114 MBytes   958 Mbits/sec
[ 10]   3.00-4.00   sec  72.1 MBytes   606 Mbits/sec
[ 12]   3.00-4.00   sec   116 MBytes   976 Mbits/sec
[ 14]   3.00-4.00   sec   114 MBytes   959 Mbits/sec
[ 16]   3.00-4.00   sec   114 MBytes   954 Mbits/sec
[ 18]   3.00-4.00   sec  72.2 MBytes   607 Mbits/sec
[SUM]   3.00-4.00   sec   834 MBytes  7.00 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   4.00-5.00   sec   105 MBytes   884 Mbits/sec
[  6]   4.00-5.00   sec   114 MBytes   957 Mbits/sec
[  8]   4.00-5.00   sec  74.0 MBytes   621 Mbits/sec
[ 10]   4.00-5.00   sec   113 MBytes   946 Mbits/sec
[ 12]   4.00-5.00   sec   114 MBytes   957 Mbits/sec
[ 14]   4.00-5.00   sec   112 MBytes   938 Mbits/sec
[ 16]   4.00-5.00   sec   112 MBytes   935 Mbits/sec
[ 18]   4.00-5.00   sec   113 MBytes   950 Mbits/sec
[SUM]   4.00-5.00   sec   857 MBytes  7.19 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   5.00-6.00   sec  80.0 MBytes   671 Mbits/sec
[  6]   5.00-6.00   sec   110 MBytes   927 Mbits/sec
[  8]   5.00-6.00   sec   111 MBytes   932 Mbits/sec
[ 10]   5.00-6.00   sec   112 MBytes   941 Mbits/sec
[ 12]   5.00-6.00   sec   110 MBytes   922 Mbits/sec
[ 14]   5.00-6.00   sec   112 MBytes   937 Mbits/sec
[ 16]   5.00-6.00   sec   111 MBytes   931 Mbits/sec
[ 18]   5.00-6.00   sec   110 MBytes   925 Mbits/sec
[SUM]   5.00-6.00   sec   857 MBytes  7.19 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   6.00-7.00   sec   112 MBytes   945 Mbits/sec
[  6]   6.00-7.00   sec   112 MBytes   938 Mbits/sec
[  8]   6.00-7.00   sec   111 MBytes   933 Mbits/sec
[ 10]   6.00-7.00   sec   112 MBytes   944 Mbits/sec
[ 12]   6.00-7.00   sec   112 MBytes   937 Mbits/sec
[ 14]   6.00-7.00   sec   112 MBytes   941 Mbits/sec
[ 16]   6.00-7.00   sec  73.1 MBytes   614 Mbits/sec
[ 18]   6.00-7.00   sec   111 MBytes   929 Mbits/sec
[SUM]   6.00-7.00   sec   855 MBytes  7.18 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   7.00-8.00   sec   112 MBytes   935 Mbits/sec
[  6]   7.00-8.00   sec   115 MBytes   962 Mbits/sec
[  8]   7.00-8.00   sec   114 MBytes   956 Mbits/sec
[ 10]   7.00-8.00   sec   112 MBytes   941 Mbits/sec
[ 12]   7.00-8.00   sec   114 MBytes   956 Mbits/sec
[ 14]   7.00-8.00   sec   111 MBytes   933 Mbits/sec
[ 16]   7.00-8.00   sec   110 MBytes   926 Mbits/sec
[ 18]   7.00-8.00   sec  73.5 MBytes   616 Mbits/sec
[SUM]   7.00-8.00   sec   862 MBytes  7.22 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   8.00-9.00   sec   114 MBytes   954 Mbits/sec
[  6]   8.00-9.00   sec   116 MBytes   976 Mbits/sec
[  8]   8.00-9.00   sec  76.4 MBytes   641 Mbits/sec
[ 10]   8.00-9.00   sec   113 MBytes   951 Mbits/sec
[ 12]   8.00-9.00   sec   116 MBytes   971 Mbits/sec
[ 14]   8.00-9.00   sec   114 MBytes   955 Mbits/sec
[ 16]   8.00-9.00   sec   113 MBytes   946 Mbits/sec
[ 18]   8.00-9.00   sec   108 MBytes   907 Mbits/sec
[SUM]   8.00-9.00   sec   870 MBytes  7.30 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   9.00-10.00  sec   116 MBytes   975 Mbits/sec
[  6]   9.00-10.00  sec   117 MBytes   979 Mbits/sec
[  8]   9.00-10.00  sec   116 MBytes   975 Mbits/sec
[ 10]   9.00-10.00  sec   115 MBytes   966 Mbits/sec
[ 12]   9.00-10.00  sec   116 MBytes   970 Mbits/sec
[ 14]   9.00-10.00  sec  90.4 MBytes   758 Mbits/sec
[ 16]   9.00-10.00  sec   114 MBytes   960 Mbits/sec
[ 18]   9.00-10.00  sec  83.0 MBytes   696 Mbits/sec
[SUM]   9.00-10.00  sec   868 MBytes  7.28 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  1.06 GBytes   911 Mbits/sec                  sender
[  4]   0.00-10.00  sec  1.06 GBytes   911 Mbits/sec                  receiver
[  6]   0.00-10.00  sec  1.07 GBytes   919 Mbits/sec                  sender
[  6]   0.00-10.00  sec  1.07 GBytes   919 Mbits/sec                  receiver
[  8]   0.00-10.00  sec  1016 MBytes   852 Mbits/sec                  sender
[  8]   0.00-10.00  sec  1016 MBytes   852 Mbits/sec                  receiver
[ 10]   0.00-10.00  sec  1.01 GBytes   869 Mbits/sec                  sender
[ 10]   0.00-10.00  sec  1.01 GBytes   869 Mbits/sec                  receiver
[ 12]   0.00-10.00  sec  1.07 GBytes   918 Mbits/sec                  sender
[ 12]   0.00-10.00  sec  1.07 GBytes   918 Mbits/sec                  receiver
[ 14]   0.00-10.00  sec  1.07 GBytes   916 Mbits/sec                  sender
[ 14]   0.00-10.00  sec  1.07 GBytes   916 Mbits/sec                  receiver
[ 16]   0.00-10.00  sec  1.05 GBytes   900 Mbits/sec                  sender
[ 16]   0.00-10.00  sec  1.05 GBytes   900 Mbits/sec                  receiver
[ 18]   0.00-10.00  sec   960 MBytes   805 Mbits/sec                  sender
[ 18]   0.00-10.00  sec   960 MBytes   805 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  8.25 GBytes  7.09 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  8.25 GBytes  7.09 Gbits/sec                  receiver

iperf Done.
*edit
Changed 10gig ports to mtu9216 and set my workstation and freenas to mtu 9000
That made single thread hit 10gig...

Code:
c:\iperf>iperf3 -c 192.168.254.200
Connecting to host 192.168.254.200, port 5201
[  4] local 192.168.254.1 port 59620 connected to 192.168.254.200 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  1.08 GBytes  9.27 Gbits/sec
[  4]   1.00-2.00   sec  1.12 GBytes  9.65 Gbits/sec
[  4]   2.00-3.00   sec  1.12 GBytes  9.63 Gbits/sec
[  4]   3.00-4.00   sec  1.13 GBytes  9.68 Gbits/sec
[  4]   4.00-5.00   sec  1.12 GBytes  9.60 Gbits/sec
[  4]   5.00-6.00   sec  1.11 GBytes  9.55 Gbits/sec
[  4]   6.00-7.00   sec  1.13 GBytes  9.70 Gbits/sec
[  4]   7.00-8.00   sec  1.11 GBytes  9.57 Gbits/sec
[  4]   8.00-9.00   sec  1.13 GBytes  9.74 Gbits/sec
[  4]   9.00-10.00  sec  1.11 GBytes  9.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  11.2 GBytes  9.60 Gbits/sec                  sender
[  4]   0.00-10.00  sec  11.2 GBytes  9.60 Gbits/sec                  receiver

iperf Done.
 
Last edited:
  • Love
Reactions: Samir

ske4za

Member
Feb 4, 2019
78
43
18
My S2500-48P has been running solid as a TOR switch with switching 10G trunks as well as POE for my cameras, so I picked up another S2500-48T to use it's 4x10G ports on my SAN side.

However after a few days the 10G ports become flaky and drop connections. Power cycling will fix it for a few more days, but randomly it'll just decide to become flaky again. I mainly got it to help me transition to the new storage node with minimal downtime, but even with just two ports being used (the storage node and my proxmox VM server) it still has the issue. I updated to the latest firmware as well. I took it out of line since it's just two units and am just running a direct DAC now, but I tried multiple ports as well as switching between different DACs and transceivers.

It's too late to return it to the seller but it is frustrating. When I have time I'll try to troubleshoot a little bit deeper, but from what I can remember it tended to happen during my scheduled backup hours so it must've needed some I/O to trigger.
 
  • Like
Reactions: Samir

AveryFreeman

consummate homelabber
Mar 17, 2017
411
52
28
41
Near Seattle
averyfreeman.com
Serious question how would everybody rate this switch (S2500 All Models) vs say the Microtik CRS226-24G-2S+ ?
I would put Aruba ahead in terms of "Enterprise Class" vs Microtik but the Microtik is much newer so its like saying I have a GTX 980 (top dog in the day) vs a GT 1060 now (low end) but the GT1060 is so much newer it beats the pants off an old card.

You think that applies to switches?

I am sure the Microtik is quieter, and uses less power, and from what I can see has many of the same features. The main reason I went with this was 4x SFP+ ports and it was cheaper. However the Microtik is NEW not used.

The other big reason is because I read about some issues on Microtik with speed loss due to a buffer issue when going from 10gb to 1gb.

I am going to get rid of my Ubiquiti US-24 it just is not feature rich enough for me and that will almost pay for both of these switches I got.
Still was keeping an eye on the Microtik however also the newer 4x SFP+ mini switch they have.
Sorry to bump necrothread, but I think issue with switchos/routeros mikrotik systems is they do all their switching through the CPU/SoC now to save costs instead of supplying an ASIC like "classic" switches. It has its advantages - lower power consumption, less money, more versatile. It also isn't as fast. If an ASIC is designed to switch packets for 4 ports at 9.4Gbps, it'll do it without even touching the CPU. If you introduce the CPU into the mix, there's a chance you'll encounter a bottleneck - as you said, either in a buffer register, poorly engineered l2 cache, or just a cheap CPU to begin with. I'm sure they'll eventually be better someday, but that day has not come yet afaik.
 
  • Like
Reactions: Samir

tjk

Active Member
Mar 3, 2013
406
140
43
www.servercentral.com
I haven't seen anything on how deep the buffers are on Miktorik switches when they mix 1/10/25/100G interfaces. Anyone know? This matters on a busy switch for sure.
 
  • Like
Reactions: Samir

AveryFreeman

consummate homelabber
Mar 17, 2017
411
52
28
41
Near Seattle
averyfreeman.com
This isn't true. Only L3 routing goes through the CPU for low-end Mikrotiks. Switching uses an ASIC.
Thanks, I appreciate the correction - will refrain from trash talking mikrotik from now on

Any examples of what differentiates a "low end" mikrotik? I was thinking anything that can be switched from "SwitchOS" to "RouterOS", which basically implies it's all being done in CPU territory
 
  • Like
Reactions: Samir

klui

Well-Known Member
Feb 3, 2019
583
276
63
Understandable but note Brocade ICXes can use switching or routing OS and L3 routing use the ASIC.

I don't run Mikrotiks but it looks like their regular switches don't support hardware routing. You need to look at their Ethernet Routers. Just check out each model's test results. Compare routing performance with their switches.

I just spot-checked a couple like the CCR2004-16G-2S+.
 

AveryFreeman

consummate homelabber
Mar 17, 2017
411
52
28
41
Near Seattle
averyfreeman.com
I just spot-checked a couple like the CCR2004-16G-2S+.
Nice, hadn't noticed they have test results on their literature. I am not trying to trash Mikrotik, I think they fill a very important space.

I was actually thinking about moving from 40GbE to 10GbE w/ RDMA just because most QSFP+ switches are damn loud, and volume is beginning to be a priority. This switch they have caught my eye in that regard: MikroTik

I was told Mellanox SX60x6 switches are OK though.
 
  • Like
Reactions: Samir