US MELLANOX SX6036 - $200

solon

Member
Apr 1, 2021
53
3
8
Never mind that last, some a bit of duckduckgo with the original fan number FFB0412UHN eventually turned up that it's a product from Delta in Thailand. For future reference:

Red: +12V
Black: GND
Blue: RPM
Yellow: PWM

That allowed me to translate it to industry-standard 4 pin on the Noctua.
 

solon

Member
Apr 1, 2021
53
3
8
I had the same problem. I plugged the unit straight into a linux pc from the mgmt1 port and ran:

Code:
sudo tcpdump --interface eth1
Then I could see the IP address that was broadcasting packets was 10.22.13.3 - and I reset my ip address to be 1 up from that in the same class D and got into the web pages with admin/admin to update the firmware.
I've just finally sat down with this. I'm no expert on tcpdump, but I'm getting:

Code:
IP 0.0.0.0.bootpc >
It seems to be making mdns requests, probably looking for a network boot? Is anyone familiar with this behavious by any chance?
 

aboron

New Member
Jul 19, 2020
4
2
3
I've just finally sat down with this. I'm no expert on tcpdump, but I'm getting:

Code:
IP 0.0.0.0.bootpc >
It seems to be making mdns requests, probably looking for a network boot? Is anyone familiar with this behavious by any chance?
Have you set a temporary static ip on your laptop/pc (anything is fine)? If not, that may just be your own machine looking for an IP address. If this is the mellanox, then it is already looking for dhcp on it's own.

When tcpdump prints "bootpc" it just means port 68 - which is the client side of dhcp (which is how you can ask for a network boot server as well as ip address).
 

solon

Member
Apr 1, 2021
53
3
8
Thanks for the quick reply.

It was doing this both with and without an address set on my ethernet port that's directly connected to the mellanox. MAC address is also the Mellanox's.

Not fully understanding why it isn't receiving an adress when I connect it on my network. There must be some restriction to what acceptable dhcp settings are. I suppose I'll just have to get a serial cable to find out what's going on.
 

saratoga

New Member
Nov 5, 2021
7
2
3
I'm interested in getting a QSFP+ ethernet switch with at least 5 ports to replace my SFP+ unit, so I was looking at these:


Just to make sure I understand, I would need to update this with the HP firmware from the March 2012 to a newer build. Then I would need to get an ethernet license. Is that right? For basic testing and ethernet use how new of a firmware do I need to update to from HP?
 

bytebuilder

New Member
Mar 19, 2018
9
1
3
I'm interested in getting a QSFP+ ethernet switch with at least 5 ports to replace my SFP+ unit, so I was looking at these:


Just to make sure I understand, I would need to update this with the HP firmware from the March 2012 to a newer build. Then I would need to get an ethernet license. Is that right? For basic testing and ethernet use how new of a firmware do I need to update to from HP?
As a quick side note about this specific listing, I purchased one from him last week and it's in fine working order, as much as I can tell so far without testing all of the ports. Awesome seller too. I too found this thread and landed on getting an SX6036 since they well priced. From your question I believe that is the process. The firmware that came with mine is SX_3.3.5006 2014-05-20 12:19:44 ppc and based on this thread am expecting a bit of an upgrade grind. But that's part of the fun of home labbing :)
 

thetoad

Active Member
Feb 10, 2021
228
89
28
I think i started doing the grind, then just skipped to the end, as got tired of it and thing seemed to work fine. :)
 

saratoga

New Member
Nov 5, 2021
7
2
3
The cover on mine seems fairly stuck. Looking at the instructions I'm supposed to take out the screws (done), remove the power supplies and fan modules (done), and then pull it back (won't budge). I got a couple people over to pull on it and it seems really stuck. Is there some trick to opening them that I'm missing?
 

Rand__

Well-Known Member
Mar 6, 2014
6,068
1,445
113
You tried prying /twisting /turning with a screwdriver?
That usually works for me;)
 

RedX1

Active Member
Aug 11, 2017
109
101
43
Hello



I had the same difficulty when first using these switches.

I would make sure that you have not missed the screw that is sometimes hidden under a small round security sticker, placed on the top cover, above the power connector in the LH side.



I hope that helps.



Good luck.



RedX1
 

saratoga

New Member
Nov 5, 2021
7
2
3
Dumb question, but is there some way to route from the 10/40G side through the MGMT RJ45 interfaces? I've got a laser printer I want on the network and I am looking to eventually get rid of my old switch (currently serving only the printer).
 

TrumanHW

Member
Sep 16, 2018
141
21
18
This project looks pretty interesting (and difficult) ... Do you think (in retrospect) IB was worth the effort ..?

I'm pretty much a networking ignoramus ... but I was also a total clue re: FN / TN before I jumped in and thus, learned a ton!
Am I correct in thinking the IB upside is:

It's an ultra low-latency, proprietary protocol which is great...
Unless Macs use that network, in which case it'd be limited to IB over Ethernet..?

Furthermore, MacOS lacks support for ROCE nor RDMA.

Workarounds..?
- Use MacOS as a HyperVisor to run a VM which supports IB and has Mellanox drivers like Windows 11 (I think) or Linux.
- If a VM can't work, add an OS "on metal" on a small partition & save files to an ExFat formatted SSD (so both OS can R/W to the dev)..?
Another benefit to Linux (AFAIK) it allows modification of Thunderbolt 3's reserved bandwidth & thus might exceed the 22Gb MacOS limit.
 

solon

Member
Apr 1, 2021
53
3
8
Not sure whether you were reacting to my project, and this is a little bit of a late reply, but I'm happy with the little infiniband network I'm running. I'm only using it as access to my network drives at the moment, but I am running games off of m.2's in my server which deliver data at around 2Gb/sec, which is more than adequate and 20x as fast as the 1gbe can do. I have a iscsi drive that looks local to my windows VM which I have for gaming.

The main upside of infiniband is the rdma I think. CPU load on the iscsi provider is negligible, which is mainly really nice when you want to move storage away from the client machine.

I'm vaguely considering moving the a whole windows install to a iscsi setup, but that would only really start to be interesting if I wanted to boot the whole VM on multiple machines. A solution that's looking for a problem at the moment, other than that it'd scratch my "it can be done so it must be done" itch. I will probably do it that way when I move to new hardware. At the moment the limiting factor is that there isn't enough pcie space for sufficient m.2 storage in my server for all my needs if I want full bandwidth for each drive.

I'm finally getting around to trying to get my 40gbe set up. Also not terribly neccesary or anything it keeps me off the street.
 

solon

Member
Apr 1, 2021
53
3
8
So, unfortunately, despite specifically looking for and finding a SX6012 with no trace of EMC markings anywhere on the box, it appears that this does in fact have a EMC uboot image... guess I'm off to find that thread about switching from EMC to regular MLNX-OS.

Code:
* Chasis Type        : DINGO
* Number of Ports    : 12
* U-Boot Revision    :U-Boot 2009.01 SX_PPC_M460EX SX_3.2.0330-82-EMC ppc (Feb 27 2013 - 12:13:42)
* Firmware Revision  : 9.9.1260
* INI file Revision  : 0x31010016
Strangely though, nothing I do seems to be making it possible to connect to the unit over SSH or making anything like a webinterface accessible.