Mellanox SX6036, Rail Kit

[DRW]

I'm not sure if these are particular amazing deals, but they seemed pretty decent to me.

SX6036 (real Mellanox, not EMC), $120

Mellanox SX6036 36 Port QSFP Managed InfiniBand Switch 2xPsu - Same Day Shipping | eBay

How do we refurbish items at our facility?.

www.ebay.com

I offered 117 and they accepted (I know, I really lowballed them there).
It comes with a 1 yr Allstate warranty, for whatever that's worth (probably not much).
Arrived pretty well packaged, but the rack ears were bent up (I straightened them again with a pair of channel-locks and very little effort).
Had some extremely sticky tape residue on it in a couple places, but it cleaned off just fine with solvent.
Overall very clean inside and not dented or anything.
I haven't gotten around to booting it up because my serial console usb cable decided to die on me and I'm waiting for a replacement.

MSX60-SKIT (Mellanox rail kit), $40 (or less if you are ordering more)

*NEW* Mellanox MSX60-SKIT, rev: A1, 1U Rails for SX60xx (Contact for Bulk Buy) | eBay

Find many great new & used options and get the best deals for *NEW* Mellanox MSX60-SKIT, rev: A1, 1U Rails for SX60xx (Contact for Bulk Buy) at the best online prices at eBay! Free shipping for many products!

www.ebay.com

It looks like this guy has a ton of these and wants to get rid of them.
I ordered one set. I can't imagine that I'd need more as I'd probably go to 100G before I max out the 6036.

 

[ribroc]

It's still tough to say no to the 6036, right? Even with low-watt 100G like that tiny mikrotik on the market I struggle a bit on the value proposition for something that's really nice but also really quiet. Spending 3x as much for 100G vs 40G, and 1/2 the power draw is very tempting if you don't need all the ports especially when breakout cables are an option.

I still think the 6036 is the best option for most people, 40G/56G is still hard enough to saturate in a home setting, and the 30-40W consumption with a quiet fan mod means you're out the door for about $200 and a weekend.

 

[DRW]

Yeah that's pretty much it. I really don't need 100G for anything atm. By the time I do, it will be even cheaper (and maybe we will be talking about 200G being tempting). Either way, I will probably still want a lot of 10G. I don't really envision this leaving my rack anytime in the next 5 years.

 

[GhettoSuperstar]

My question for everyone, do I need this? I have a desktop and a workstation.

Both are powerful but the workstation is really overkill. I want to use the workstation for AI training and such. I tried to use a Bluefield ConnectX-5 100G cards but the desktop used the card without issue. The workstation didn't (don't know why but it's a server board, go figure). Nothing I could do, could get the Bluefield ConnectX-5 cards recognized by the motherboard (Gigabyte MZ32-AR0 rev.1).

So i sold the Bluefield cards and bought ConnectX-4 cards PCIe 16x. Both computers recognize the cards without issue, took the firmware without issue. But for the life of me, I can't get OpenSM working so that I can have computer to computer direct infiniband connection. I could buy this and rely on 40/56g infiniband speeds without relying on OpenSM. Or I could just settle for 100g Ethernet direct communications. I already have all the 100g ethenet fiber tranceivers and it just works. Sorry for the epic story.

 

[jbv1982]

I just got mine today. The ears were bent (I imagine people sitting the switch on its side) but I bought the rail kit too so I can replace them. Now to get it unlocked to ethernet mode.

 

[DRW]

I just got mine today. The ears were bent (I imagine people sitting the switch on its side) but I bought the rail kit too so I can replace them. Now to get it unlocked to ethernet mode.

Keep all the little screws from the ears. The new rail kit doesn't seem to include those (or at least mine didn't). Just fyi, in case you haven't received the rails yet.

My switch was on v3.4.2008 btw.

 

[jbv1982]

Mine did come with screws for the ears. They were in a tiny bag in either the big screw bag or the cage nut bag, I can't remember which.

 

[DRW]

I just checked again, just in case. I didn't get any. My rail box was fairly beat-up so it's possible the bag fell out on the way here (rails were fine though).

 

[Digital Spaceport]

It's incredibly hard to saturate 100gbe connections with single stream eth based traffic. 25gbe is rather expensive. 10gbe is not fast enough. 40gbe is still a really sweet spot on performance/price.

 

[NablaSquaredG]

I just checked again, just in case. I didn't get any. My rail box was fairly beat-up so it's possible the bag fell out on the way here (rails were fine though).

Screws are simply UNC 6-32, but as far as I remember with a 100° countersink angle, not the standard 82°

 

[unphased]

that tiny mikrotik on the market I struggle a bit on the value proposition for something that's really nice but also really quiet. Spending 3x as much for 100G vs 40G, and 1/2 the power draw is very tempting if you don't need all the ports especially when breakout cables are an option.

As a 6036 owner I’m intrigued by your notion that this mikrotik 100G unit is only 3x as much. By what I can see you’re talking about the 4 port 100G Mikrotik switch which is $800. $150 or $120 for a 6036 is significantly cheaper than one third of 800. It is a factor of 5. Even if it were a factor of 3 it is hard to justify, although if a real use case existed it is easier. Either way, being sub 200 makes it justifiable to get a 6036 to switch from point to point networks to a proper switched network. I benefit from my 40Gbit switch by getting thunderbolt networking to my Apple devices via my CX4 card. I just wouldn’t have been able to properly route the internet through my workstation to deliver it this way if I was using point to point.

 

[Cruzader]

As a 6036 owner I’m intrigued by your notion that this mikrotik 100G unit is only 3x as much. By what I can see you’re talking about the 4 port 100G Mikrotik switch which is $800. $150 or $120 for a 6036 is significantly cheaper than one third of 800. It is a factor of 5.

He could simply not be US based.
In the European market id pay less for the mikrotik 100g 4port than 3x sx6036.

 

[iron-bound]

Cheaper Restored Mellanox SX6036 36 Port QSFP Managed InfiniBand Switch 2xPsu (Refubished)

 

[ribroc]

I’m intrigued by your notion that this mikrotik 100G unit is only 3x as much. By what I can see you’re talking about the 4 port 100G Mikrotik switch which is $800.

He could simply not be US based.
In the European market id pay less for the mikrotik 100g 4port than 3x sx6036.

I can't speak to other markets, however you can commonly buy the MikroTik CRS504-4XQ-IN via amazon for below $660USD and the prices have fluctuated. For my own mental comparison for a quiet switch, I was including the rails, and the 3-4 fans used to mod for quiet comfort. Which as I said, would be about $200 and a weekend. I did not mean to say that $14.95 (wow) or $120 for the 6036 alone was fully comparable, but I did base that off my own assumed ~$650 price for the mikrotik.

 

[Nnyan]

I really need more 10Gb ports and was considering the 8-12 port options out there but if this is relatively low power use and can be made to run quiet I am really considering it.

 

[unphased]

It’s fair... here’s hoping 100G gear drops more, yeah I guess I’ll keep an eye on that mikrotik. Low noise is a big deal. I got really into playing with my 6036 but … it’s actually not that easy to mod to be quiet. I replaced fans, which, yeah it takes a while to do, i did a CPU heatsink replacement, which turned out pretty unnecessary especially as my fan situation still isn’t sorted and it isn’t quiet yet.

 

[Cruzader]

There are some sn2100 100g units listed domesticly here at equivalent of 515€ and 600€, somewhat tempted to grab one.
Last i looked on ebay they were still closer to 1500 there.

Ive been a bit stuck on the "il keep using my 6036 intil i can get a cheap 100g replacement" but 515€ should be getting into cheap-ish.

 

[klui]

SN2100s are prone to the Atom AVR54 errata, so choose carefully. Even though they're power misers their fans are loud so for those adverse to "loud" noises they need to be modded unless you install Cumulus and change the fan thresholds.

 

[iron-bound]

Given the rush to buy GPU nodes for AI and the massive bandwidth required to scale. I'd expect a bunch of 100G gear to hit the market at some point as people upgrade

 

[Koop]

real Mellanox, not EMC

Damn chill.... They already got bought out leave em alone... Now they gotta be FAKERS too?