Aruba MAS series SFP+ & POE+ switches sub-$100

[pr1malr8ge]

I'm an idiot, and did not know that you needed to add threads to Iperf3 to get higher bitrates.

c:\iperf>iperf3 -P 8 -c 192.168.254.200
Connecting to host 192.168.254.200, port 5201
[  4] local 192.168.254.1 port 53756 connected to 192.168.254.200 port 5201
[  6] local 192.168.254.1 port 53757 connected to 192.168.254.200 port 5201
[  8] local 192.168.254.1 port 53758 connected to 192.168.254.200 port 5201
[ 10] local 192.168.254.1 port 53759 connected to 192.168.254.200 port 5201
[ 12] local 192.168.254.1 port 53760 connected to 192.168.254.200 port 5201
[ 14] local 192.168.254.1 port 53761 connected to 192.168.254.200 port 5201
[ 16] local 192.168.254.1 port 53762 connected to 192.168.254.200 port 5201
[ 18] local 192.168.254.1 port 53763 connected to 192.168.254.200 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   100 MBytes   842 Mbits/sec
[  6]   0.00-1.00   sec   102 MBytes   851 Mbits/sec
[  8]   0.00-1.00   sec  66.0 MBytes   553 Mbits/sec
[ 10]   0.00-1.00   sec   100 MBytes   840 Mbits/sec
[ 12]   0.00-1.00   sec   101 MBytes   849 Mbits/sec
[ 14]   0.00-1.00   sec   100 MBytes   838 Mbits/sec
[ 16]   0.00-1.00   sec  99.2 MBytes   832 Mbits/sec
[ 18]   0.00-1.00   sec   100 MBytes   843 Mbits/sec
[SUM]   0.00-1.00   sec   769 MBytes  6.45 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   1.00-2.00   sec   110 MBytes   920 Mbits/sec
[  6]   1.00-2.00   sec  73.8 MBytes   619 Mbits/sec
[  8]   1.00-2.00   sec   113 MBytes   950 Mbits/sec
[ 10]   1.00-2.00   sec   109 MBytes   918 Mbits/sec
[ 12]   1.00-2.00   sec   110 MBytes   924 Mbits/sec
[ 14]   1.00-2.00   sec   109 MBytes   911 Mbits/sec
[ 16]   1.00-2.00   sec   108 MBytes   910 Mbits/sec
[ 18]   1.00-2.00   sec   111 MBytes   934 Mbits/sec
[SUM]   1.00-2.00   sec   845 MBytes  7.09 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   2.00-3.00   sec   120 MBytes  1.01 Gbits/sec
[  6]   2.00-3.00   sec   121 MBytes  1.01 Gbits/sec
[  8]   2.00-3.00   sec   120 MBytes  1.00 Gbits/sec
[ 10]   2.00-3.00   sec  76.5 MBytes   641 Mbits/sec
[ 12]   2.00-3.00   sec  86.0 MBytes   720 Mbits/sec
[ 14]   2.00-3.00   sec   118 MBytes   987 Mbits/sec
[ 16]   2.00-3.00   sec   118 MBytes   986 Mbits/sec
[ 18]   2.00-3.00   sec  77.4 MBytes   648 Mbits/sec
[SUM]   2.00-3.00   sec   837 MBytes  7.01 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   3.00-4.00   sec   116 MBytes   971 Mbits/sec
[  6]   3.00-4.00   sec   116 MBytes   970 Mbits/sec
[  8]   3.00-4.00   sec   114 MBytes   958 Mbits/sec
[ 10]   3.00-4.00   sec  72.1 MBytes   606 Mbits/sec
[ 12]   3.00-4.00   sec   116 MBytes   976 Mbits/sec
[ 14]   3.00-4.00   sec   114 MBytes   959 Mbits/sec
[ 16]   3.00-4.00   sec   114 MBytes   954 Mbits/sec
[ 18]   3.00-4.00   sec  72.2 MBytes   607 Mbits/sec
[SUM]   3.00-4.00   sec   834 MBytes  7.00 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   4.00-5.00   sec   105 MBytes   884 Mbits/sec
[  6]   4.00-5.00   sec   114 MBytes   957 Mbits/sec
[  8]   4.00-5.00   sec  74.0 MBytes   621 Mbits/sec
[ 10]   4.00-5.00   sec   113 MBytes   946 Mbits/sec
[ 12]   4.00-5.00   sec   114 MBytes   957 Mbits/sec
[ 14]   4.00-5.00   sec   112 MBytes   938 Mbits/sec
[ 16]   4.00-5.00   sec   112 MBytes   935 Mbits/sec
[ 18]   4.00-5.00   sec   113 MBytes   950 Mbits/sec
[SUM]   4.00-5.00   sec   857 MBytes  7.19 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   5.00-6.00   sec  80.0 MBytes   671 Mbits/sec
[  6]   5.00-6.00   sec   110 MBytes   927 Mbits/sec
[  8]   5.00-6.00   sec   111 MBytes   932 Mbits/sec
[ 10]   5.00-6.00   sec   112 MBytes   941 Mbits/sec
[ 12]   5.00-6.00   sec   110 MBytes   922 Mbits/sec
[ 14]   5.00-6.00   sec   112 MBytes   937 Mbits/sec
[ 16]   5.00-6.00   sec   111 MBytes   931 Mbits/sec
[ 18]   5.00-6.00   sec   110 MBytes   925 Mbits/sec
[SUM]   5.00-6.00   sec   857 MBytes  7.19 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   6.00-7.00   sec   112 MBytes   945 Mbits/sec
[  6]   6.00-7.00   sec   112 MBytes   938 Mbits/sec
[  8]   6.00-7.00   sec   111 MBytes   933 Mbits/sec
[ 10]   6.00-7.00   sec   112 MBytes   944 Mbits/sec
[ 12]   6.00-7.00   sec   112 MBytes   937 Mbits/sec
[ 14]   6.00-7.00   sec   112 MBytes   941 Mbits/sec
[ 16]   6.00-7.00   sec  73.1 MBytes   614 Mbits/sec
[ 18]   6.00-7.00   sec   111 MBytes   929 Mbits/sec
[SUM]   6.00-7.00   sec   855 MBytes  7.18 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   7.00-8.00   sec   112 MBytes   935 Mbits/sec
[  6]   7.00-8.00   sec   115 MBytes   962 Mbits/sec
[  8]   7.00-8.00   sec   114 MBytes   956 Mbits/sec
[ 10]   7.00-8.00   sec   112 MBytes   941 Mbits/sec
[ 12]   7.00-8.00   sec   114 MBytes   956 Mbits/sec
[ 14]   7.00-8.00   sec   111 MBytes   933 Mbits/sec
[ 16]   7.00-8.00   sec   110 MBytes   926 Mbits/sec
[ 18]   7.00-8.00   sec  73.5 MBytes   616 Mbits/sec
[SUM]   7.00-8.00   sec   862 MBytes  7.22 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   8.00-9.00   sec   114 MBytes   954 Mbits/sec
[  6]   8.00-9.00   sec   116 MBytes   976 Mbits/sec
[  8]   8.00-9.00   sec  76.4 MBytes   641 Mbits/sec
[ 10]   8.00-9.00   sec   113 MBytes   951 Mbits/sec
[ 12]   8.00-9.00   sec   116 MBytes   971 Mbits/sec
[ 14]   8.00-9.00   sec   114 MBytes   955 Mbits/sec
[ 16]   8.00-9.00   sec   113 MBytes   946 Mbits/sec
[ 18]   8.00-9.00   sec   108 MBytes   907 Mbits/sec
[SUM]   8.00-9.00   sec   870 MBytes  7.30 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   9.00-10.00  sec   116 MBytes   975 Mbits/sec
[  6]   9.00-10.00  sec   117 MBytes   979 Mbits/sec
[  8]   9.00-10.00  sec   116 MBytes   975 Mbits/sec
[ 10]   9.00-10.00  sec   115 MBytes   966 Mbits/sec
[ 12]   9.00-10.00  sec   116 MBytes   970 Mbits/sec
[ 14]   9.00-10.00  sec  90.4 MBytes   758 Mbits/sec
[ 16]   9.00-10.00  sec   114 MBytes   960 Mbits/sec
[ 18]   9.00-10.00  sec  83.0 MBytes   696 Mbits/sec
[SUM]   9.00-10.00  sec   868 MBytes  7.28 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  1.06 GBytes   911 Mbits/sec                  sender
[  4]   0.00-10.00  sec  1.06 GBytes   911 Mbits/sec                  receiver
[  6]   0.00-10.00  sec  1.07 GBytes   919 Mbits/sec                  sender
[  6]   0.00-10.00  sec  1.07 GBytes   919 Mbits/sec                  receiver
[  8]   0.00-10.00  sec  1016 MBytes   852 Mbits/sec                  sender
[  8]   0.00-10.00  sec  1016 MBytes   852 Mbits/sec                  receiver
[ 10]   0.00-10.00  sec  1.01 GBytes   869 Mbits/sec                  sender
[ 10]   0.00-10.00  sec  1.01 GBytes   869 Mbits/sec                  receiver
[ 12]   0.00-10.00  sec  1.07 GBytes   918 Mbits/sec                  sender
[ 12]   0.00-10.00  sec  1.07 GBytes   918 Mbits/sec                  receiver
[ 14]   0.00-10.00  sec  1.07 GBytes   916 Mbits/sec                  sender
[ 14]   0.00-10.00  sec  1.07 GBytes   916 Mbits/sec                  receiver
[ 16]   0.00-10.00  sec  1.05 GBytes   900 Mbits/sec                  sender
[ 16]   0.00-10.00  sec  1.05 GBytes   900 Mbits/sec                  receiver
[ 18]   0.00-10.00  sec   960 MBytes   805 Mbits/sec                  sender
[ 18]   0.00-10.00  sec   960 MBytes   805 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  8.25 GBytes  7.09 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  8.25 GBytes  7.09 Gbits/sec                  receiver

iperf Done.

*edit
Changed 10gig ports to mtu9216 and set my workstation and freenas to mtu 9000
That made single thread hit 10gig...

c:\iperf>iperf3 -c 192.168.254.200
Connecting to host 192.168.254.200, port 5201
[  4] local 192.168.254.1 port 59620 connected to 192.168.254.200 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  1.08 GBytes  9.27 Gbits/sec
[  4]   1.00-2.00   sec  1.12 GBytes  9.65 Gbits/sec
[  4]   2.00-3.00   sec  1.12 GBytes  9.63 Gbits/sec
[  4]   3.00-4.00   sec  1.13 GBytes  9.68 Gbits/sec
[  4]   4.00-5.00   sec  1.12 GBytes  9.60 Gbits/sec
[  4]   5.00-6.00   sec  1.11 GBytes  9.55 Gbits/sec
[  4]   6.00-7.00   sec  1.13 GBytes  9.70 Gbits/sec
[  4]   7.00-8.00   sec  1.11 GBytes  9.57 Gbits/sec
[  4]   8.00-9.00   sec  1.13 GBytes  9.74 Gbits/sec
[  4]   9.00-10.00  sec  1.11 GBytes  9.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  11.2 GBytes  9.60 Gbits/sec                  sender
[  4]   0.00-10.00  sec  11.2 GBytes  9.60 Gbits/sec                  receiver

iperf Done.

 

[ske4za]

My S2500-48P has been running solid as a TOR switch with switching 10G trunks as well as POE for my cameras, so I picked up another S2500-48T to use it's 4x10G ports on my SAN side.

However after a few days the 10G ports become flaky and drop connections. Power cycling will fix it for a few more days, but randomly it'll just decide to become flaky again. I mainly got it to help me transition to the new storage node with minimal downtime, but even with just two ports being used (the storage node and my proxmox VM server) it still has the issue. I updated to the latest firmware as well. I took it out of line since it's just two units and am just running a direct DAC now, but I tried multiple ports as well as switching between different DACs and transceivers.

It's too late to return it to the seller but it is frustrating. When I have time I'll try to troubleshoot a little bit deeper, but from what I can remember it tended to happen during my scheduled backup hours so it must've needed some I/O to trigger.

 

[AveryFreeman]

Serious question how would everybody rate this switch (S2500 All Models) vs say the Microtik CRS226-24G-2S+ ?
I would put Aruba ahead in terms of "Enterprise Class" vs Microtik but the Microtik is much newer so its like saying I have a GTX 980 (top dog in the day) vs a GT 1060 now (low end) but the GT1060 is so much newer it beats the pants off an old card.

You think that applies to switches?

I am sure the Microtik is quieter, and uses less power, and from what I can see has many of the same features. The main reason I went with this was 4x SFP+ ports and it was cheaper. However the Microtik is NEW not used.

The other big reason is because I read about some issues on Microtik with speed loss due to a buffer issue when going from 10gb to 1gb.

I am going to get rid of my Ubiquiti US-24 it just is not feature rich enough for me and that will almost pay for both of these switches I got.
Still was keeping an eye on the Microtik however also the newer 4x SFP+ mini switch they have.

Sorry to bump necrothread, but I think issue with switchos/routeros mikrotik systems is they do all their switching through the CPU/SoC now to save costs instead of supplying an ASIC like "classic" switches. It has its advantages - lower power consumption, less money, more versatile. It also isn't as fast. If an ASIC is designed to switch packets for 4 ports at 9.4Gbps, it'll do it without even touching the CPU. If you introduce the CPU into the mix, there's a chance you'll encounter a bottleneck - as you said, either in a buffer register, poorly engineered l2 cache, or just a cheap CPU to begin with. I'm sure they'll eventually be better someday, but that day has not come yet afaik.

 

[tjk]

I haven't seen anything on how deep the buffers are on Miktorik switches when they mix 1/10/25/100G interfaces. Anyone know? This matters on a busy switch for sure.

 

[klui]

Sorry to bump necrothread, but I think issue with switchos/routeros mikrotik systems is they do all their switching through the CPU/SoC now to save costs instead of supplying an ASIC like "classic" switches.

This isn't true. Only L3 routing goes through the CPU for low-end Mikrotiks. Switching uses an ASIC.

 

[AveryFreeman]

This isn't true. Only L3 routing goes through the CPU for low-end Mikrotiks. Switching uses an ASIC.

Thanks, I appreciate the correction - will refrain from trash talking mikrotik from now on

Any examples of what differentiates a "low end" mikrotik? I was thinking anything that can be switched from "SwitchOS" to "RouterOS", which basically implies it's all being done in CPU territory

 

[klui]

Understandable but note Brocade ICXes can use switching or routing OS and L3 routing use the ASIC.

I don't run Mikrotiks but it looks like their regular switches don't support hardware routing. You need to look at their Ethernet Routers. Just check out each model's test results. Compare routing performance with their switches.

I just spot-checked a couple like the CCR2004-16G-2S+.

 

[AveryFreeman]

I just spot-checked a couple like the CCR2004-16G-2S+.

Nice, hadn't noticed they have test results on their literature. I am not trying to trash Mikrotik, I think they fill a very important space.

I was actually thinking about moving from 40GbE to 10GbE w/ RDMA just because most QSFP+ switches are damn loud, and volume is beginning to be a priority. This switch they have caught my eye in that regard: MikroTik

I was told Mellanox SX60x6 switches are OK though.

 

[sic0048]

Has anyone done true layer 3 routing with Vlans, acl, and DHCP on a S2500 switch? I currently run my Vlans though a pfSense firewall, but I'm going to be redoing my network soon and want to go with all of that on the switch. I also have a Brocade 6450 switch, but my understanding is those switches are not good at DHCP because the can't run as an authoritative DHCP, so lots of devices cannot get a DHCP assignment. That leaves me with trying it all on my existing S2500.

 

[cryonix]

Anyone has the latest firmware for this? I am unable to find it in the drivers & firmware search area on the HPE website. I have an account there. I made a support ticket, but no response yet.

 

[cryonix]

@klui fancy running into you here, you replied to my post about DAC compatibility issue on the Aruba MAS SX500 switch on reddit.

 

[klui]

Try this link instead when you're logged in.

HPE Networking | Enterprise

networkingsupport.hpe.com

 

[Sogndal94]

Anyone has the latest firmware for this? I am unable to find it in the drivers & firmware search area on the HPE website. I have an account there. I made a support ticket, but no response yet.

View attachment 38073

i can get you ArubaOS_MAS_7.4.0.7_80725 for s2500-24p, if that is what you are looking for.

 

[cryonix]

Oh wow, that worked, thanks @klui and @Sogndal94 thanks for offering the Zip too, but I was able to find it via the link klui provided. I have a question though, the versioning seems a bit strange to me it seems 7.4.0.7 is the newest firmware, but the version before that is 7.4.1.12? Maybe I should read the SemVer docs again.

And on another note, I have decided to just order Aruba & Intel coded DAC cable from FS to use with this switch since that's the cheapest, brand new DAC cable that I have found (researching online) to be working on this switch.

Edit: According to this test bed data sheet on FS, 202405231512440l2ebb.pdf (fs.com), The S2500 is definitely supported if you choose Aruba coding for any of their DAC/Transceivers, since it is included in their test bed. Which seems to suggest that any of the MAS SX500 switches would also be supported.
So just wanted to link that here, personally I think this is much cheaper than getting them from HPE or used JWXXXA DAC cables. If anyone finds a cheaper alternative, do let me know. Again, appreciate the help guys, and I hope this helps someone else too.

 

[cryonix]

Hey again guys,

I’m looking for a comprehensive guide on setting up VLANs with trunking on the Aruba S2500 switch. The command syntax is a bit different from Cisco, and since the S2500 was initially designed to work with a controller, there are some unique steps involved.

I’ve managed to set up VLANs with trunking, but I’m encountering an issue where the switch is routing traffic internally, rather than passing it to my firewall (OPNsense) for routing based on my firewall rules.

Here’s what I’ve done so far:

• Created VLANs and assigned them to ports.
• Configured a trunk port to handle VLAN traffic.

However, the switch seems to be handling routing on its own, and I want to ensure that all inter-VLAN routing is managed by my firewall.

Could someone provide a step-by-step guide or point me to a resource that explains how to properly configure this setup so that the routing is offloaded to the firewall?

Thanks in advance for your help!

Edit-1: I was able to find this link Aruba S2500 Definitive Guide to VLANs : r/homelab (reddit.com), I'll give it a try and update here if it works. That guide attempts to use the GUI to setup everything, but I'll try to achieve exactly what he did using the CLI so I can copy paste the commands here if anyone else is interested.

 

[cryonix]

Hi, I am still working on figuring out the best way to deal with VLANs on this switch, in different topologies, and management, I'll post my findings later. But I wanted to quickly put an update here.

I talked with FS customer support, and they reported that, after testing the cable, this was the cheapest ($11) DAC they have, that works with Aruba. They tested it between Aruba S2500 and an Intel server network adapter. The reason I mention is this because, if you choose custom coding for Aruba on the transceiver, it costs double, so you can just get the premade cables from the link I posted, which they said works.

Intel XDACBL1M 10G SFP+ DAC Twinax Cable - FS.com

So, as far as I know, this is the cheapest, brand new DAC cable you can get for this switch, that is known for working. I just placed my order today. Hope this helps someone.