EXPIRED Mellanox InfiniBand SX6018 - Switch - EMC only - $300 OBO

[necr]

Got 4xNoctua NF-A4x20 and replaced 4 super noisy stock fans out of HELL.
Default pinout is as follows (you 're looking at terminals, with two smaller hooks facing down, holes facing away):
1.Red +12v
2.Blue Tachometer(RPM)
3.Yellow PWM
4.Black GND

I've unmounted stock fan block, removed wires from white connectors, removed black connectors from Noctua fans, connected wires as above. Below is Noctua color wiring, same layout
1.Yellow +12v
2.Green RPM
3.Blue PWM
4.Black GND

After connecting all of them to board PWM takes effect (without active alarms fans go from ~20% to ~40% in cycles, which makes it unbearable). Even with Noctua low-noise adapter (which kinda makes cycles smoother) it was more irritating that running with 100% speed.
So, I removed PWM wire from each connector. System detects all fans, but can't control them, and it's really quiet!
So, my advice would be to get 3-pin fans and wire them the same way so you would know if one of them really fails.
It's probably not possible to control fans from PPC board, tc daemon just reads data there.

If that helped, please endorse: https://goo.gl/RfjbnG

 

[herby]

Since this thread had a little more fan-mod talk than the Beware of EMC switches sold as Mellanox SX6XXX on eBay thread I thought I'd ask here. Does anyone know what the slowest fan RPMs the SX6012 will tolerate without alarm?

edit:
About 2 months after this original post I swapped out the two Yate Loon D80SM-12 80mm fans I modded piggyback style on the switch for two Sanyo 9S0812P4F01 fans. I split the tach lead to two headers each; so the switch reads it like the two fans are four. These fans are PMW, 2800 RPM and reading 2600 - 2700ish in MLNX-OS. No fan alarms with this setup and CPU_BOARD_MONITOR2 temps in the low 60s.

 

[Freebsd1976]

Since this thread had a little more fan-mod talk than the Beware of EMC switches sold as Mellanox SX6XXX on eBay thread I thought I'd ask here. Does anyone know what the slowest fan RPMs the SX6012 will tolerate without alarm?

did you found the slowset fan RPMs?
at least delta efb0412hhd 8200rpm 9.43cfm 0.15amp pwm fan is not enough, first it will set 60% after boot up,then it will 40% 100% and then 40% 100% loops(in sx6012 log )
after I cut pwm wire. everything is ok , fan light remain green , no alerts messages . The temperature is 5 degrees higher than origin fan.
Will try 9200rpm 11000rpm pwm 13000 rpm fan if I have time

 

[Bjorn Smith]

Hi,

An update - I have done the above alteration with Noctua 40x25mm non PWM fans - and its awesome - my switch is "silent" - as in there is only a silght hum of the fans, nothing you will hear if you are in a room with any kind of noise - and certainly something I could fall asleep to in case I had to put my switch in my bedroom.

I bought 5 fans and switched out my fans, not using screws to make then stick, but strips - 4 in the fan compartment and one in the PSU that I have mounted - if my PSU dies I will do the same with the backup PSU.

Just a word of advice.

The color scheme mentioned by @necr are wrong for the non pwm fans.

It is much easier, since the colors are the same.

Yellow -> Yellow (RPM)
Red -> Red (12V)
Black -> Black (Ground)

So Noctua NF-A4x20 FLX - is the way to go for a silent switch

My temps after a while running idle with one port connected:

Temperature
Module Component Temperature [°C] Status
MGMT BOARD_MONITOR 31.00 OK
MGMT CPU_BOARD_MONITOR 40.00 OK
MGMT CPU_BOARD_MONITOR2 75.00 OK
MGMT QSFP_TEMP1 27.00 OK
MGMT QSFP_TEMP2 29.50 OK
MGMT QSFP_TEMP3 34.00 OK
MGMT SX 32.00 OK

Module Device Fan Speed [RPM] Status
FAN FAN F1 5010.0 OK
FAN FAN F2 5010.0 OK
FAN FAN F3 5070.0 OK
FAN FAN F4 4920.0 OK
PS1 FAN - - NOT PRESENT
PS2 FAN F1 5190.0 OK

So far so good - I will add more servers to the switch, but will probably only reach 4 or 5 connected servers, since I only intent to use the switch as my storage/vmotion backent.

Power usage is also a bonus with using less crazy fans:

Power Status
Module Device Sensor Power[W] Voltage[V] Current[A] Status
MGMT CURR_MONITOR 12V 33.95 11.73 2.89 OK
Total Power Summary
Total power used:   33.95 Watts

 

[necr]

The color scheme mentioned by @necr are wrong for the non pwm fans.

It is much easier, since the colors are the same.

Yellow -> Yellow (RPM)
Red -> Red (12V)
Black -> Black (Ground)
[/CODE]

Nice temps! Seeing that you have 5 fans, you probably have SX6036 where pinout may differ. My notes were for SX6012(converted EMC).

 

[deff]

I've got one of those cheap unmanaged MSX6790 for my homelab.
Almost had no problem with upgrading stock fans with noctua's 40x20mm ones. But I can't solve the puzzle with PSU fan which produces 60dBA at 23000 RPM. I've tried to solder in fan emulator, tuned to original RPM, but it didn't do. Looks like PSU checks several points of fan RPM.

 

[deff]

Small update to my previous post.
I could make PSU work with Noctua 40x20mm fan and fan emulator tuned to 23600 RPM. Left the second PSU in it's original state in case my mod would cause a fire.

 

[Mighty Man]

Small update to my previous post.
I could make PSU work with Noctua 40x20mm fan and fan emulator tuned to 23600 RPM. Left the second PSU in it's original state in case my mod would cause a fire.

Got pictures?

 

[deff]

Didn't make photos at first, was too happy to see it not falling into fault state. Now, after ~12 hours of smooth running (w/o any traffic) in a rack, i removed PSU and took some photos.

Some notes:
1. PSU fan mounting screws are too small for Noctua holes while Nostua screws do not go through chassis mounting holes. I used rubber mounting pins.
2. Electrical connection goes as follow:
Fan yellow wire -> fan emu +12V -> PSU (red)
Fan black wire -> fan emu GND -> PSU (black)
fan emu (output) -> PSU (blue)
Other fan and PSU wires are not connected.

 

[Mighty Man]

Wow, that's pretty amazing! Looks great!

 

[cyrus104]

Awesome work, which fan emulator did you use? Also what happens when you don't use a fan emulator on the PSU fans?

 

[deff]

Mine are similar to these, but there is a catch - you need to tweak them to original RPM. I'm not sure what are the failure limits of PSU controller but if you stay close to original fan RPM (23000 +-8%), you're safe. These emulators are based on 555 timer IC, whose output parameters are defined by a capacitor (C) and a couple resistors (R1 and R2). When you get your emulators, you should measure values of C, R1 and R2, put them in any online 555 timer calc and adjust value a resistor to get output frequency ~196Hz (+-8%). Order those parts and solder them to your emulator.
There are 555 based generators with adjustable resistor like these, you need one that covers desired output frequency.

Probably I should add a disclaimer. Here it goes:
The PSU manufacturer put those noisy fans for a reason, they are much higher performing than Noctua replacement parts. I've never tested modified PSU under full load and I can't guarantee they will perform well.

 

[soccrstar]

I did the same thing Bjorn Smith did and ran 6x flx noctua. 4x in fan chassis and 1 in each power supply. Slight hum but I can sleep in peace with it on. Full speed 100% of the time.

I don't run fully loaded and never will and doubt noctua fans will hold at fully loaded. I have 7 ports active and air coming from psu are warm and never was noticeably warm with factory fans.

7 ports out of 36. I don't recommend swapping PSU fans of any fans for that matter if you going to decently load it especially for sustained periods of time.

http://imgur.com/a/YcPKBwp

 

[NablaSquaredG]

Anyone know what the difference between the variants with different airflow direction is (i.e.. PSU -> Connector vs Connector -> PSU)?

It seems like the PSU -> Connector airflow variants are only available as standard depth whereas the Connector -> PSU variants are always short depth.

I recently ordered a MSX6036F-1BRR (i.e. short depth, Connector->PSU), but I'd like to have PSU -> Connector airflow direction (I'm going to mount this thing with the connectors facing the back of the rack because I don't want to route the passive direct attach QSFP cables from the front to the back and there are no QSFP patch panels).

Do you reckon it's going to work if I just take PSU + Fan module, turn all the fans and give it a go?

 

[Rand__]

Never tried it, but I can tell you that the differetn fan/psu modules are compatible, so I just bought the desired airflow version and switched mine.
Given that they often can be had for ok prices ($30 or so) that seemed way less hassle

 

[int0x2e]

Never tried it, but I can tell you that the differetn fan/psu modules are compatible, so I just bought the desired airflow version and switched mine.
Given that they often can be had for ok prices ($30 or so) that seemed way less hassle

Just going to say that if you don't mind a bit of work, you should be able to open up the PSUs and fan modules and flip the fans around so the airflow is reversed without buying another unit.

 

[Bjorn Smith]

Just going to say that if you don't mind a bit of work, you should be able to open up the PSUs and fan modules and flip the fans around so the airflow is reversed without buying another unit.

That would be easily doable, since I exchanged the entire fan for a noctua one - so just turning the fan around would be "easy" compared.

 

[kaz]

Inspired by this thread, I am trying to replace the fans of my SX6036F-1BRR with Noctua NF-A4x20 FLX (non-PWM) as attached pictures.
The 4-pin connector of the original fans and on the small board is small, and the 3-pin connector of NF-A4x20 FLX does not fit into the connector on the small board.
I am trying to find some cable with such a small 4-pin connector, but I cannot find them because it is difficult to differentiate a small connector from the normal connector.
Does somebody know how to call such a small 4-pin connector?

 

[deff]

Does somebody know how to call such a small 4-pin connector?

It's Molex Picoblade.

 

[kaz]

It's Molex Picoblade.

deff, thanks for your quick answer.
I cannot find Molex PicoBlade, and Molex PicoBlade seems different from JST-GH, even though both have 1.25mm pitch

JST connector - Wikipedia

en.wikipedia.org

but I will be trying Amazon.com: Readytosky MX1.25 Pitch Connectors and Pre-Crimped Cables Silicone 15cm Wire for APM2.6 APM2.8 Pixhawk Flight Controller Runcam Camera: Toys & Games
and I will update how it goes later.