US MELLANOX SX6036 - $200

[aboron]

Here's a copy of the MLNX-OS version upgrade chart from HP's site - knowing this plus your current version will help plot an upgrade path.



Seems like this could be a painful, multiple-step process depending on how old your starting point.
I just ordered one of these today, since my 8 port IS5022 is already full.

 

[ViciousXUSMC]

Those running these at home, are you mostly breaking it out to 10gb fiber? Tempting to get one to play with, but I cant image the actual cost once I buy all the adapters, breakouts, etc.

 

[StevenDTX]

Those running these at home, are you mostly breaking it out to 10gb fiber? Tempting to get one to play with, but I cant image the actual cost once I buy all the adapters, breakouts, etc.

The only breakout Im using is to connect to my 10GbE switches. Everything else is 40GbE.

 

[RedX1]

Seems like this could be a painful, multiple-step process depending on how old your starting point.
I just ordered one of these today, since my 8 port IS5022 is already full.

Hi


This can be a very stressful process and will depend on the age of the currently installed Firmware image.

Please make sure you have plenty of time before you start this process.


I started one upgrade from SX_PPC_M460EX SX_3.3.4100 2013-09-16 18:27:29 ppc and took that all the way to the Lenovo PPC_M460EX 3.6.8012 2019-02-22 07:53:42 ppc.

From memory that was around a 6 or maybe 8 stage process. I used the upgrade train information from the HP site - you have indicated


The switch takes around 5 minutes to fully boot.

There are some gotcha’s using the update train in the Web GUI – please see Post 20 in this thread.

It takes around 10-15 minutes to load the next stage HP Firmware and then around 25 minutes to fully install it. You then need to re-boot the switch.

The process is described here.

Please be patient, there are long wait times - where the switch has no indication of progress.


The upgrade from, the HP 3.6.8010 to the Lenovo PPC_M460EX 3.6.8012 2019-02-22 07:53:42 ppc. takes much more time - maybe around 120 minutes.

I thought this last upgrade process had failed; I did not interrupt it - finally the Web GUI succeeded.



Once you have the latest firmware you can look and enabling the features you want to use.



Good luck.





RedX1

 

[jwest5637]

Wow, it strikes me as super fortunate that I found this thread!
I am coincidentally in the thick of the firmware upgrade process, having just failed several attempts before finding HPE's upgrade chart.

@Rand__
Could you expand on the Ethernet conversion you mentioned (for me and any other uninitiated)?

@cw823
Similarly, I get the sense you can guide on ETH access?

Edit:
I found in another thread @cw823 noting to reach out if anyone needed help with ETH license troubles, he had me sorted in no time.
Thanks @cw823!

 

[solon]

Hey all, I'm considering getting a SX6012 as internet speeds above 1gbit have just become possible in my area, but I'm a little concerned about the likelihood of getting 40gbe working on it. I already have a 40gb infiniband SAN using an IS5022, but I thought it would be more interesting to have a 40gbe ethernet next to it rather than going with a more main stream home networking solution.

I have attached an image of the partnumber of the unit I'm considering getting (e-bay), I'd love to know if anyone has had any success getting these working as 40gbe units?

 

[cw823]

40Gb License is easy, HMU for assistance.

 

[jwest5637]

+1 for @cw823, who helped me out with the same.

 

[solon]

Cool. I'll see about getting ahold of one of these things then. Thanks so far!

 

[solon]

I have the one from the image on the way. Shipping and VAT are almost as much as the device itself, but I really couldn't find anything else available. I'll send some PM's when it comes in. It'd be really neat if this all works!

 

[solon]

Not sure if this is completely the right topic for this, but alot of this Mellanox router stuff seems to be concentrated here, so here goes...

This thing came in. It's a bit of a screamer, which will require some adjustments. Unfortunately, it's not requesting an adress over DHCP. Does anyone know of a way to access it other that through the RS232 port? I'd have to go out and buy an adapter solely for that, which I'll do if I must, but if there's another way that anyone knows of I'l love to hear it. The documentation that I've found all seems to suggest the RS232 port is the way to do it, but I was wondering what the function of the USB port is.

Also, the unit clearly isn't quite as 'new' as the seller claimed, it has a sticker on it proclaiming it to be property of the Brazilian national telecommunications provider and obviously someone has switched DHCP off on the RJ45 ethernet port.

Anyway, I think I'll probably make 200mm holes in top and bottom and get some 200mm Noctua's to take it under 15db. Did something similar with my IS5022, but that had a route for internal airflow that allowed me to use only one fan and a 3D printed airguide. See if they purposely changed the connector pinout to fry people's industry standard fans on this one the same way they did on the IS5022.

 

[jwest5637]

Not sure if this is completely the right topic for this, but alot of this Mellanox router stuff seems to be concentrated here, so here goes...

This thing came in. It's a bit of a screamer, which will require some adjustments. Unfortunately, it's not requesting an adress over DHCP. Does anyone know of a way to access it other that through the RS232 port? I'd have to go out and buy an adapter solely for that, which I'll do if I must, but if there's another way that anyone knows of I'l love to hear it. The documentation that I've found all seems to suggest the RS232 port is the way to do it, but I was wondering what the function of the USB port is.

Also, the unit clearly isn't quite as 'new' as the seller claimed, it has a sticker on it proclaiming it to be property of the Brazilian national telecommunications provider and obviously someone has switched DHCP off on the RJ45 ethernet port.

Anyway, I think I'll probably make 200mm holes in top and bottom and get some 200mm Noctua's to take it under 15db. Did something similar with my IS5022, but that had a route for internal airflow that allowed me to use only one fan and a 3D printed airguide. See if they purposely changed the connector pinout to fry people's industry standard fans on this one the same way they did on the IS5022.

I also had to buy an adapter, but it's been useful to have on hand for other peices of hardware too. I recommend just buying a cheap "USB console cable" (<$20).

The manual for these is pretty easy to find online, and there is a factory reset button too. But I beleive DHCP is off by default, and has to be enabled via console, if I remember correctly.

No idea on the fans, but I like your idea of swapping them out, they are a bit noisy. (and blowing air in the wrong direction for my setup.)

 

[klui]

Not sure if this is completely the right topic for this, but alot of this Mellanox router stuff seems to be concentrated here, so here goes...

No, for the item you purchased that would be this thread.

I've never tried to see if the EMC software actually passed 40G traffic. Has anyone else?

Best practice for any kind of enterprise gear is to connect to the console to ensure it boots up properly.

 

[solon]

I'm relatively sure mine is not an EMC unit. It just needs licensing. Oh, and a cable. need to order a cable to see what's going on with the dhcp and get it doing 40gbe.

 

[thetoad]

yes, I'm pretty sure I posted what the static ip was, my serial console port didn't work either. I found the ip via wireshark, i'd need to go through my post history to find what it set the static ip too

 

[thetoad]

https://forums.servethehome.com/index.php?threads/mellanox-default-ip-address.32770/

 

[aboron]

I'm relatively sure mine is not an EMC unit. It just needs licensing. Oh, and a cable. need to order a cable to see what's going on with the dhcp and get it doing 40gbe.

I had the same problem. I plugged the unit straight into a linux pc from the mgmt1 port and ran:

sudo tcpdump --interface eth1

Then I could see the IP address that was broadcasting packets was 10.22.13.3 - and I reset my ip address to be 1 up from that in the same class D and got into the web pages with admin/admin to update the firmware.

 

[solon]

This seems like an excellent suggestion. I haven't actually gotten around to doing anything else to mine, still need to order my noctua's to make it silent.

I'll give this a try, would certainly simplify matters.

 

[StevenDTX]

This console cable works too: https://smile.amazon.com/gp/product/B01AFNBC3K

 

[solon]

Finally cut a 200mm hole in the lid of my sx6012. Two things became clear. One, the lid functions as a bit of a heat sink for the psu's. I don't imagine this will be a problem with the fan directly above it, (unless those pads are purely for insulation) but I may need to turn the fan round so that the highest velocity air hits the psu's first, but we'll see.

In any event, the pinout on the Mellanox fans seems to be of the "he, he, we'll fry anything non-mellanox you plug in" variety, just like in the IS5022. It has red-blue-yellow-black instead of the expected black-yellow-green-blue as is the customary colour coding. hopefully black and yellow are +12V and ground, just like in the IS5022. Question then is which is pwm and which is rpm? Don't suppose anyone here happens to know for mellanox fans? I don't really know enough about electronics to know what to measure to distinguish the two.