EU Mellanox SX6036 36ports QSFP €150

550i

New Member
Apr 19, 2019
2
0
1
EU
Oh, and I almost forgot to say thanks @Rand__ ! I bought 3pcs thanks to you already in April and now I improved my homelab to 56GbE.
 
Nov 23, 2014
90
29
18
Is there any difference between the MSX6036T-1BRS and MSX6036F-2SRS aside from depth and the number of PSUs? I'm mostly concerned about power consumption in Ethernet configuration. These days the standard length, 2PSU variant is much easier to get. Do theses switches accept non-mellanox sfp+/qsfp+? It seems LR4 might be an issue while most people use DACs.
 
Last edited:

Rand__

Well-Known Member
Mar 6, 2014
4,558
909
113
http://www.mellanox.com/related-docs/prod_ib_switch_systems/PB_SX6036.pdf

MSX6036T-1BRS
SwitchX®-2 based 36-port QSFP FDR10 1U Managed InfiniBand switch system with a non-blocking switching capacity of 4Tb/s. 1PS, short depth, C2P airflow, Rail Kit, RoHS6

MSX6036F-2SRS
SwitchX®-2 based 36-port QSFP FDR 1U Managed InfiniBand switch system with a non-blocking switching capacity of 4Tb/s. 2PS, standard depth, C2P airflow, Rail Kit, RoHS6

So different speed...
The 2 psu unit will use more power due to the second psu and its fan.


Cables- i also use DACs and found that with 40g IB cables you might fall down to 10g ETH.
FDR cables are not that expensive though, but didnt look for optics tbh.
 
  • Like
Reactions: RageBone
Nov 23, 2014
90
29
18
http://www.mellanox.com/related-docs/prod_ib_switch_systems/PB_SX6036.pdf
So different speed...
The 2 psu unit will use more power due to the second psu and its fan.


Cables- i also use DACs and found that with 40g IB cables you might fall down to 10g ETH.
FDR cables are not that expensive though, but didnt look for optics tbh.
My bad, I meant to compare the one you originally posted (MSX6036F-1BRS) to the standard depth, 2PSU version (they are much easier to find these days) :confused:
Also, I should have been more clear regarding the PSUs. A second PSU naturally consumes additional power, however it can always be removed.
For me it's a decision between one of the 36Ports models and perhaps the 6012, which might just be enough for my use case. However, power consumption doesn't seem to differ that much.

I have a ton on SFP+ modules (mostly brocade from FS.com) and some mellanox qsfp+ to sfp+ adapters, it would be nice if they were compatible.
 

Rand__

Well-Known Member
Mar 6, 2014
4,558
909
113
Well the 36 port model uses significantly more power than my 12 port model ( 30 vs 50 or, not actual numbers, but i posted both before here on one thread or the other [this one and the conversion thread most likely]).

I have used mlx qsfp sfp+ adapter and a dell dac sfp+, that was no issue, havent used any other.
 
  • Like
Reactions: Psycho_Robotico

Takrbark3

New Member
Dec 17, 2017
18
0
1
39
Does somebody has a list what feature licenses are available for this SX1036/SX6036 platform?
EFM_SX: Generix SX license
OEM_LIC: 10G/25G/100G license

Other?
 

Renat

New Member
Jun 8, 2016
6
5
3
37
Now you can just install options for its; no old non-upgraded software and long way to convert.

-L2 (and "full" in old naming)
-L3
-GW
-Fabric inspector


Takrbark3, MLNX-OS is a global OS so 10G/25G/100G is not for SX6036, only for next revisions.
 
Last edited:

Rand__

Well-Known Member
Mar 6, 2014
4,558
909
113

cw823

Active Member
Jan 14, 2014
274
79
28
Took way longer to get the SX6036 to the latest firmware than it did to get the ethernet license. Thanks to a couple folks that helped point me the right direction.

I did NOT need the conversion so simply needed to create the dummy license, watch the call via ltrace, grab the shared secret, and create the key. Will keep the ubuntu VM around that I used for this in case someone else needs assistance.
 

kh78

New Member
Mar 31, 2020
27
6
3
Well, I have 3 SX6036s on the way. Didn't actually find the idea of these via this thread, but it has been a fortuitous discovery after the fact, good info!

Once I get the switches and check they power on happily etc, I might need to seek some help on the OS and feature set side, but I will see how I go.

Can anyone recommend suitable 56Gbe fibre transceivers (ideally LC but not essential) for these switches?
I'll need to mix copper and fibre.

Any dual port, 56Gbe IB/EN capable NiCs that people have had success with?
 
Last edited:

Rand__

Well-Known Member
Mar 6, 2014
4,558
909
113
If you got genuine ones then OS shouldnt be an issue, fw links above.

Have not used transceivers on mine (DAC only) but I believe MLX are not locked, so most should work fine.

The goto card for these would be a (branded) CX356, flashed to native MLX for ~$20 ea - various threads about these here too.
 
  • Like
Reactions: kh78

joddlarn

Member
Jul 14, 2018
43
15
8
Sweden
I can just add to what @Rand__ wrote. I've used the same NICs and only DACs.
What I know, there are no 56Gbe optical modules. If you want to use optical modules you have to go with 40Gbe. (if I'm wrong here, please correct me).

But for the DACs, my SX6036 has been a bit picky. I bought a few 40G ones from fs.com and they didn't work. The switch logged an error about the cables not being supported. They worked fine with Arista and Wedge switches (after re-flashing the cables with their "fsbox"). I contacted FS and they told me that they had some compatibility issues with Mellanox and the version of the cables that i was using. They sent me new ones and they worked just fine.
 
  • Like
Reactions: kh78

kh78

New Member
Mar 31, 2020
27
6
3
Okay thanks guys, great forum community here!

What you are saying, that backs up the fact that I couldn't find any QSFP14 56Gb optical transceivers, only QSFP28 or DAC. For the most part I can get away with DACs, just one backbone trunk that I need to run on optical so if it costs a bit more for that one, then so be it I guess. Thanks for the warning on the FS.com DACs, I'll be wary of that.

Are QSFP28 Optical transcievers backwards compatible to run at 56G IB/EN if needed?