Aruba MAS series SFP+ & POE+ switches sub-$100

Arbies · Apr 26, 2020

n17ikh said:
Which coding did you get for the ipolex SFP+? Intel?

Yep the Intel.

They must be doing something funny in these modules. b/c Aruba reports them as fiber

(ArubaS2500-48P) #show interface gigabitethernet 0/1/0 transceiver

Vendor Name : Intel Corp
Vendor Serial Number : INSSRK30461
Vendor Part Number : FTLX8571D3BCV-I
Aruba Certified : NO
Cable Type : 10GBASE-SR
Connector Type : LC
Wave Length : 850 nm

but the transceiver seem to be happy at 10G

(ArubaS2500-48P) #show interface gigabitethernet 0/1/1

GE0/1/1 is administratively Up, Link is Up, Line protocol is Up
Hardware is Gigabit Ethernet, Interface is GE0/1/1, Address is 00:0b:86:ab:2a:b3
Encapsulation ARPA, Loopback not set
Configured: duplex (Full), Speed (10 Gbps), FC (Off), Autoneg (Off)
Interface index: 130
MTU 1514 bytes
Link flaps: 2
Flags: Access, Trusted
Link status last changed: 2d 17:19:39 ago
Last update of counters: 0d 00:00:06 ago
Last clearing of counters: 1w1d 02:15:40 ago
Statistics:
Received 117634980 frames, 138723376876 octets
40.741 Kpps, 20.931 Mbps
117461653 unicast, 55688 multicast, 117639 broadcast
0 runts, 0 giants, 0 throttles
0 error octets, 0 CRC frames
Transmitted 105155162 frames, 82873201385 octets
81.304 Kpps, 987.236 Mbps
103215764 unicast, 1183200 multicast, 756198 broadcast
0 throttles, 0 errors octets, 0 deferred
0 collisions, 0 late collisions
Interface is not PoE capable

Jiaxinxi · May 2, 2020

Just found another issue today. It might be by design.

It seems like this switch has to be connected to a router directly. If it is after another dumb switch, it won't be able to connect to the network.

Is this normal?

Jiaxinxi · May 3, 2020

I changed to this fan

Coolerguys 40mm (40x40x20) 12v Ultra Quiet Fan CG4020L12S for Pi Devices, 3D Printers, Arduino

It is a lot cheaper and super quiet too. 1/2 of the price of a Noctua fan

It is a two pin fan, so there is no RPM reading. The front fan status light will blink red.

Code:

Show inventory

The switch itself is pretty hot. It is uncomfortable to touch.

For those who made fan changes, how hot is your switch? Is it uncomfortable to touch?

ske4za · May 4, 2020

@Jiaxinxi mine sits in a rack in the garage and it's starting to get hot outside, so mine is warm too. Mine made it through a Southern summer last year in the garage which was probably close to 100F ambient during the hot parts of the day. Happy to report no casualties other than a failed SS in my pfSense, but that might be more to it being a cheap SSD (60gb something or other) than the heat.

Code:

 show poe controller

Linecard  Power Budget(W)  Power Consumption(W)  GuardBand(mW)  PoE Management  Cisco Compatibility  Config Delay(ms)
--------  ---------------  --------------------  -------------  --------------  -------------------  ----------------
0         400              18                    11000          Dynamic         Disabled             2000

Code:

show inventory

Show Inventory
--------------


System Card Slot              : 0
SC Serial #                   : xxxx (Date: 10/07/12)
SC Model Name                 : xxxxx
Mgmt Port HW MAC Addr         : xxxx
HW MAC Addr                   :xxxx
CPLD Version                  : Rev: 2.4 (0x2.4)
PoE Firmware Version          : 4.1.5 (Build: 1)
CPU Assembly #                : 2010110C (Rev: 03.B0)
CPU Serial #                  : xxxxx(Date: 10/07/12)
Power Supply                  : Present (580W)
                              : 12V System Voltage Ok
                              : 56V PoE Voltage Ok
Power Supply Serial #         : xxxxx (Date: 09/01/12)
Power Supply Model No         : 2510064
Power Supply Vendor Model No  : IPSN-580APA1 (Rev: 48.48)
Console Card Assembly #       : 2010111C (Rev: 01.00)
Console Card Serial #         : xxxxx(Date: 10/07/12)
PoE Card Assembly #           : 2010106C (Rev: 01.00)
PoE Card Serial #             : xxxxx(Date: 10/07/12)
System Temperature            : CPU Temp                : 67  C
                              : DPE-0 Temp              : 90  C
                              : DPE-1 Temp              : 83  C
                              : DDR Temp                : 60  C
                              : PSU Temp0               : 49  C
                              : PSU Temp1               : 54  C
                              : Exhaust Temp            : 60  C
                              : Mid Temp                : 64  C
                              : Ambient Temp            : 54  C
                              : Console Card Temp       : 38  C
System Voltages               : 3300mV Sense            : 3360.0 mV
                              : 2500mV Sense            : 2480.0 mV
                              : 1800mV Sense            : 1808.0 mV
                              : 1000mV Sense            : 1002.0 mV
                              : 1100mV MAC              : 1098.0 mV
                              : 1200mV Sense            : 1216.0 mV
                              : 5000mV 5VDD             : 5022.0 mV
                              : 5000mV 5VSB             : 4998.0 mV
                              : 900mV VTT               : 914.0 mV
System Fan Tachometers        : Power Supply Fan 0      : 5869  RPM (Low Speed)
                              : Power Supply Fan 1      : 5869  RPM (Low Speed)
                              : FAN 0                   : 5895  RPM (Low Speed)
                              : FAN 1                   : 5696  RPM (Low Speed)

Jiaxinxi · May 4, 2020

ske4za said:

@Jiaxinxi mine sits in a rack in the garage and it's starting to get hot outside, so mine is warm too. Mine made it through a Southern summer last year in the garage which was probably close to 100F ambient during the hot parts of the day. Happy to report no casualties other than a failed SS in my pfSense, but that might be more to it being a cheap SSD (60gb something or other) than the heat.

Code:

 show poe controller

Linecard  Power Budget(W)  Power Consumption(W)  GuardBand(mW)  PoE Management  Cisco Compatibility  Config Delay(ms)
--------  ---------------  --------------------  -------------  --------------  -------------------  ----------------
0         400              18                    11000          Dynamic         Disabled             2000

Code:

show inventory

Show Inventory
--------------


System Card Slot              : 0
SC Serial #                   : xxxx (Date: 10/07/12)
SC Model Name                 : xxxxx
Mgmt Port HW MAC Addr         : xxxx
HW MAC Addr                   :xxxx
CPLD Version                  : Rev: 2.4 (0x2.4)
PoE Firmware Version          : 4.1.5 (Build: 1)
CPU Assembly #                : 2010110C (Rev: 03.B0)
CPU Serial #                  : xxxxx(Date: 10/07/12)
Power Supply                  : Present (580W)
                              : 12V System Voltage Ok
                              : 56V PoE Voltage Ok
Power Supply Serial #         : xxxxx (Date: 09/01/12)
Power Supply Model No         : 2510064
Power Supply Vendor Model No  : IPSN-580APA1 (Rev: 48.48)
Console Card Assembly #       : 2010111C (Rev: 01.00)
Console Card Serial #         : xxxxx(Date: 10/07/12)
PoE Card Assembly #           : 2010106C (Rev: 01.00)
PoE Card Serial #             : xxxxx(Date: 10/07/12)
System Temperature            : CPU Temp                : 67  C
                              : DPE-0 Temp              : 90  C
                              : DPE-1 Temp              : 83  C
                              : DDR Temp                : 60  C
                              : PSU Temp0               : 49  C
                              : PSU Temp1               : 54  C
                              : Exhaust Temp            : 60  C
                              : Mid Temp                : 64  C
                              : Ambient Temp            : 54  C
                              : Console Card Temp       : 38  C
System Voltages               : 3300mV Sense            : 3360.0 mV
                              : 2500mV Sense            : 2480.0 mV
                              : 1800mV Sense            : 1808.0 mV
                              : 1000mV Sense            : 1002.0 mV
                              : 1100mV MAC              : 1098.0 mV
                              : 1200mV Sense            : 1216.0 mV
                              : 5000mV 5VDD             : 5022.0 mV
                              : 5000mV 5VSB             : 4998.0 mV
                              : 900mV VTT               : 914.0 mV
System Fan Tachometers        : Power Supply Fan 0      : 5869  RPM (Low Speed)
                              : Power Supply Fan 1      : 5869  RPM (Low Speed)
                              : FAN 0                   : 5895  RPM (Low Speed)
                              : FAN 1                   : 5696  RPM (Low Speed)

Thanks! I am glad that your switch survived brutal summer in the south.

I am in south TX.

I have been debating about putting all the server level gears in garage. The summer here is really hot and humid. But on the other hand, the money I spent to make those servers/switch quiet is enough to buy new gear when it broke.

Not to mention regardless of the effort, the noise level will still be an issue if placed inside house.

ske4za · May 4, 2020

Jiaxinxi said:
Thanks! I am glad that your switch survived brutal summer in the south.

I am in south TX.

I have been debating about putting all the server level gears in garage. The summer here is really hot and humid. But on the other hand, the money I spent to make those servers/switch quiet is enough to buy new gear when it broke.

Not to mention regardless of the effort, the noise level will still be an issue if placed inside house.

I just moved from South Texas a couple years ago (to North Texas) but there I had my homelab in the office. It wasn't too loud when it was cold, but during the summer the fans would spool up to about white noise level and I could hear it on the other side of the house with the door open. If I closed the door, I wouldn't hear it for a few minutes, but then the room would start to heat up, and then the fans would get louder. There was no return in that room either. Where I moved up here I wired all the network cables into the garage because I didn't care about the noise out there. I'm going to try insulate the garage door and do a few other things to see if I can mitigate some of the heat during this summer, but last summer I just let it ride. Some of the units were very warm to the touch, but CPU temps seemed to be reasonable compared to the ambient, probably because most of the fans were at or near max.

For reference I attached my network rack which is above my 12U server rack that sits on the ground. Excuse the mess in there This one has two fans above the rack that blow outward. I also attached the most recent pic of the server rack where you can also see my mess on top of the rear facing S2500 (since cleaned up now!).

madmailman · May 4, 2020

I signed up just to say thanks for this thread. Going to make my life with the Aruba S2500 much better. So thanks!!

I'm still trying to work out how to run some fiber from my crawl space up into my attic. My main switch (Aruba s2500-48p) is in a small coat closet between the house and garage in the middle of the house. I've made a hole from their into the crawl space and cabled up all the downstairs rooms with CAT6 (shielded). I just need to work out the best way to get a cable from the "server room" up to the attic where I'll drop it into a closet and have another switch to connect up all the upstairs rooms. *sigh* If I was any good at fixing dry walling this would have already been done.

Another option is to run the cable out the crawl space, up the side of the house (probably in a j-channel) and back in through the soffit. Can you run fiber outdoors?

Too many questions. Anyways, hi guys!

ske4za · May 4, 2020

madmailman said:
I signed up just to say thanks for this thread. Going to make my life with the Aruba S2500 much better. So thanks!!

I'm still trying to work out how to run some fiber from my crawl space up into my attic. My main switch (Aruba s2500-48p) is in a small coat closet between the house and garage in the middle of the house. I've made a hole from their into the crawl space and cabled up all the downstairs rooms with CAT6 (shielded). I just need to work out the best way to get a cable from the "server room" up to the attic where I'll drop it into a closet and have another switch to connect up all the upstairs rooms. *sigh* If I was any good at fixing dry walling this would have already been done.

Another option is to run the cable out the crawl space, up the side of the house (probably in a j-channel) and back in through the soffit. Can you run fiber outdoors?

Too many questions. Anyways, hi guys!

I didn't want to go OT more than I already have, so I sent you a PM

madmailman · May 5, 2020

ske4za said:
I didn't want to go OT more than I already have, so I sent you a PM

Thanks for your help. I may get brave sometime soon and give this a shot. Maybe.

For everybody else if you don't already have version 7.4.1.12 for the S2500-48P you can find it on my Google Drive here. Please let me know if the link doesn't work, I'm normally trying to make sure nobody has access to my files and not sharing them.

Jiaxinxi · May 10, 2020

I made some modifications to cool the switch off and make it quieter. I have tried those fans, but it doesn't move enough air to cool the switch. After a while, the switch become uncomfortable to touch. So I tried different ways to cool it off.

I have Aruba S2500-48P. It has 4 10Gb SFP+ port. But the stock fans come with the switch is pretty noisy in a home lab environment.

There are 4 fans. 2 for the power supply, 2 for the logic board. This is the POE model I have. Non-POE model might have less fans.

This model idles at about 50W, so the solution has to be able to handle at least 50W

I changed these fan Coolerguys 40mm (40x40x20) 12v Ultra Quiet Fan CG4020L12S for Pi Devices, 3D Printers, Arduino

It is very quiet, but it barely move any air.
RODUCT SPECIFICATION:
DC Fan Size: 40x40x20mm
Bearing Type: Sleeve Bearing
Fan Speed: 3500 RPM (±10%) RPM
Noise Level: 19.50 DBA
Rated Voltage: 12 Volt
Max Airflow: 4.9 CFM

Compare to the stock fan DELTA DC FFB0412VHN
Operating Supply Voltage:12 VDC
Airflow:15.79 CFM
Speed:9500 RPM
Bearing Type:Ball
Noise:41.9 dBA
Power Rating:1.92 W

It only move about 1/3 of the air. The switch becomes uncomfortable to touch.

To fix this issue, I used a POE splitter to power 2 120MM fan to blow cool air to the switch

It is a little bit better

DPE-0 Temp drops about 10 degree

This is good enough temperature for a switch, it is designed to work in much worse environment. But since I am stuck at home, why not make some more modifications just for fun!

So I decide to change to 4 120MM fan.

Just use zip tie to tie everything down.

Put a 1u rack shelf upside down on top of the switch. It looks pretty good.
Could use some 1u rack spacer to seal the top.

The shelf is 8 inch deep. So it is not enough to cover the whole unit. With deeper shelf, it should be able to cover the whole switch.

On Rack

Now the number. After the switch runs few hours, below are the output of the

As you can see the temperature is much lower.

Temp	With Coolerguys CG4020L12S fan	With 2 120 MM fan Blow in front of switch	With top off and 4 120mm fan
CPU	70	63	45
DPE-0	95	89	68
DPE-1	81	77	63
DDR	62	55	36
PSU 0	54	52	44
PSU 1	60	58	40
Exhaust	65	57	38
Mid	70	63	42
Ambient	58	54	43
Console Card	40	37	35

For DPE-0 temperature, it is about 30 degree lower. And it is super quiet. Using iPhone app, I am able to measure the noise at 35 db right in front of the switch. Wife acceptance is increased exponentially.

Thanks for your reading!

Please refer to my blog for more detail

Decoy · Jun 6, 2020

Thanks All. I've joined the club with an S2500-48P. Fashionably late to the party - but these still seemed to be great value (despite being complete overkill in my home environment). I could have gotten away with a 24P but those listed were more expensive!

Firmware update was easy via USB. Setup was fairly intuitive - but I'm doing nothing more complicated than a bunch of VLAN and appropriate port access/trunk allocations. OPNsense handles the routing.

Noise is actually not as bad as I had expected. I've already got some Noctua's on order... so they'll go in when they eventually arrive. Also swapping out my server fans at the same time - I'm hoping for a decent step change in volume give their price relative to the hardware they'll be cooling.

Only one of my existing DACs is functional (a 7m FS.com generic). Note that an equivalent 1.5m FS.com generic was incompatible. Looking at the specs, it appears the S2500 only support 0.5m, 1, 3, 5 & 7m lengths, which may explain this? For reference, the other incompatible DAC was a Cisco SFP-H10GB-CU5M. Anyhow, I'm persisting with trying to stick with DACs over fiber so have a couple more on the way (HPE s/hand gambles!)

Edit: Quick update on the fans; My S2500-48P came with 2x Delta FFB0412SHN (13000rpm, 54.5dBA) at the rear, and 2x Delta FFB0412VHN (9500rpm, 45dBA) at the front. Before changing out, the fans reported pretty much exactly 5000rpm.

After changing to all Noctua NF-A4x20 FLX w/o low noise adapters, the reported fan speed was ~5860rpm. I have since reopened and added the low noise adapters (NA-RC10) and now have the fans reporting between 5000-5300rpm, the GUI seems to average these and reports 5180rpm. Noise is now better.

Also, FS.com have responded to my query re DAC lengths making any difference in this case with a resounding "no". They are sending another 1.5m DAC with 'different' coding to trial. Hopefully I'll have more luck this time!

myrison · Jun 13, 2020

Welcome to the club. Just a suggestion to be sure to monitor temperatures after changing the fans. I had to pull my Noctuas back out for an intermediate CFM/noise option because the temperatures were dangerously high after I swapped. Your mileage may vary but it’s worth checking temperatures to be sure you’re not running too hot.

ske4za · Jun 13, 2020

Decoy said:
Only one of my existing DACs is functional (a 7m FS.com generic). Note that an equivalent 1.5m FS.com generic was incompatible. Looking at the specs, it appears the S2500 only support 0.5m, 1, 3, 5 & 7m lengths, which may explain this? For reference, the other incompatible DAC was a Cisco SFP-H10GB-CU5M. Anyhow, I'm persisting with trying to stick with DACs over fiber so have a couple more on the way (HPE s/hand gambles!)

Did you try allow-unsupported-transceiver? I also detailed the DACs I use here https://forums.servethehome.com/ind...poe-switches-sub-100.23056/page-6#post-219554 as well as https://www.amazon.com/gp/product/B078YSWDS6/ which Emulsifide says works.

I believe in the manual, the supported DAC cables are J9281B, J9283B, and J9285B which are 1m, 3m, and 7m, respectively. I think if you can have FS code them with that they should work (maybe even other HPE/Aruba coding works).

klui · Jun 13, 2020

ske4za said:
Did you try allow-unsupported-transceiver?

The S2500s that are referenced in this thread don't support this command. It was added on ArubaOS 16.02 while S2500s and their siblings run 7.4.

Decoy · Jun 14, 2020

ske4za said:
I believe in the manual, the supported DAC cables are J9281B, J9283B, and J9285B which are 1m, 3m, and 7m, respectively.

Not sure what manual you are referring to but I can confirm that an HPE J9285B (7m) is not 'Aruba Certified' but is working none the less:

Code:

GE0/1/0
-------
Vendor Name                                : MergeOptics Gmb
Vendor Serial Number                       : CN34FRP051     
Vendor Part Number                         : 10119467-5070LF
Aruba Certified                            : NO
Cable Type                                 : 10GBASE-DAC-P
Connector Type                             : Copper Pigtail
Wave Length                                : 256 nm
Cable Length                               : 7m

A little disappointed in iperf3 speeds attained though. Currently maxing out at around 3.5Gbit/s... from Linux workstation to ESXi hosted Linux/Unix VMs. Meanwhile real world transfers (disk to disk) match this transfer speed - so I think something in the network chain is limiting throughput...

Decoy · Jun 14, 2020

myrison said:
Welcome to the club. Just a suggestion to be sure to monitor temperatures after changing the fans. I had to pull my Noctuas back out for an intermediate CFM/noise option because the temperatures were dangerously high after I swapped. Your mileage may vary but it’s worth checking temperatures to be sure you’re not running too hot.

Thanks. Yup will watch the temps pretty closely. Currently sitting at 75degC on light load ;-s.

I too trialed an intermediate solution: I used one of the Delta FFB0412VHN (9500rpm, 45dBA) fans on back and front, along with 2x Noctuas. Unfortunately it resulted in a weird pulsing frequency sound that I couldn't live with... that is when I got the additional two Noctuas.

ske4za · Jun 15, 2020

klui said:
The S2500s that are referenced in this thread don't support this command. It was added on ArubaOS 16.02 while S2500s and their siblings run 7.4.

Decoy said:
Not sure what manual you are referring to but I can confirm that an HPE J9285B (7m) is not 'Aruba Certified' but is working none the less:

Code:

GE0/1/0 ------- Vendor Name : MergeOptics Gmb Vendor Serial Number : CN34FRP051 Vendor Part Number : 10119467-5070LF Aruba Certified : NO Cable Type : 10GBASE-DAC-P Connector Type : Copper Pigtail Wave Length : 256 nm Cable Length : 7m

A little disappointed in iperf3 speeds attained though. Currently maxing out at around 3.5Gbit/s... from Linux workstation to ESXi hosted Linux/Unix VMs. Meanwhile real world transfers (disk to disk) match this transfer speed - so I think something in the network chain is limiting throughput...

I apologize, I've been hands on more with 2930s lately, as my S2500 has just been chugging along at home without any issues.

According to this page, the "Aruba Certified" DACs are mostly Molex and Accton Tech part numbers.

I'm not entirely sure those part numbers are legit, as my 3M DACs that are "Aruba Certified" are Molex 74752-1301, which is different than 74752-9992 listed above. I guess any Molex that starts with 74752 will probably be "Aruba Certified", but ymmv.

ViciousXUSMC · Jun 15, 2020

Decoy said:
Not sure what manual you are referring to but I can confirm that an HPE J9285B (7m) is not 'Aruba Certified' but is working none the less:

Code:

GE0/1/0 ------- Vendor Name : MergeOptics Gmb Vendor Serial Number : CN34FRP051 Vendor Part Number : 10119467-5070LF Aruba Certified : NO Cable Type : 10GBASE-DAC-P Connector Type : Copper Pigtail Wave Length : 256 nm Cable Length : 7m

A little disappointed in iperf3 speeds attained though. Currently maxing out at around 3.5Gbit/s... from Linux workstation to ESXi hosted Linux/Unix VMs. Meanwhile real world transfers (disk to disk) match this transfer speed - so I think something in the network chain is limiting throughput...

When I was super obsessed with IPerf results I found that I was only getting like 7/8gbps until I enabled jumbo frames then I got the full 10gbps.
I created a vlan just for my servers and anything using 10gb interfaces and put them all on Jumbo frames.

Ran it like that for a long time, just last week decided to change it back to standard 1500MTU just because in reality none of my storage was fast enough to need the extra speed, but the administrative overhead of figuring out how to configure each and every VM I had for jumbo frames was kind of a nightmare (Especially Freenas and its Jails)

The CPU can easily hold back your iperf results, and you want to make sure your using a syntax that is good for testing faster connections.
AKA you will want to run parrellel.

I used something simple like -P 10 to run 10 parallel streams.

Decoy · Jun 30, 2020

Decoy said:
A little disappointed in iperf3 speeds attained though. Currently maxing out at around 3.5Gbit/s... from Linux workstation to ESXi hosted Linux/Unix VMs. Meanwhile real world transfers (disk to disk) match this transfer speed - so I think something in the network chain is limiting throughput...

So... it turns out I had a busted NIC in my workstation. Doh!

Replaced both with s/hand HP 546sfp+ dual cards and now comfortably hit 9.4Gbit/s with 1500 MTU.

cheezehead · Jul 2, 2020

18+ months later and the thread is still kicking

SFP+ modules are SPF+ modules. Pre-HPE firmware use to be pretty open, but the newer firmware builds are "whitelisting" certain part numbers to be "supported". Unsupported, can still work just fine. If you have an SFP coding box, the validation is in the part number. After that, then there's the physical limitations of form factor (some RJ-45 adapters are bulky and could port-block neighboring ports), power draw, , thermal, and eeprom limits.

There is no allow-unsupported-transceiver command for these switches, that is an AOS (aka Procurve) command running more recent code.

For those using the 10G-Base-T SFP+ modules, have you had any luck running those at 2.5G or 5G?

Aruba MAS series SFP+ & POE+ switches sub-$100

New Member

Member

Member

Member

Member

Member

Attachments

New Member

Member

New Member

Member

New Member

Member

Member

༺༻

New Member

New Member

Member

Active Member

New Member

Active Member