Nokia WBX 210 32x-100gbe QSFP28 switch

[Bartosz Grzelak]

Hello STH

I have recently purchased the above switch from local recycling company, knowing that this switch runs ONIE I was mistakenly thinking I can install any Linux network distro, but unfortunately it's not possible, it looks like it's only supports Nuage. Did I just got a useless high-end honk of aluminum?

Does anyone know where I can get the Nuage os from? There is very little info about it.

Link to photos taken with cover off;

Microsoft OneDrive

1drv.ms

 

[NablaSquaredG]

Did I just got a useless high-end honk of aluminum?

Yep, you most probably did.

I was also eyeballing this switch together with @Rand__ but after careful consideration we decided against it.

You can run either Nuage or, as far as I know, SR OS on it. SR Linux ist not supported.

 

[Bartosz Grzelak]

Yep, you most probably did.

I was also eyeballing this switch together with @Rand__ but after careful consideration we decided against it.

You can run either Nuage or, as far as I know, SR OS on it. SR Linux ist not supported.

Hmm, yeah. Not an easy win here. Nuage license/image cost 15k haha.

I've messed about with it for few hours.
And... It turns out I have an access to uefi bios. I can boot from USB and completely omit the ONIE. I tried Ubuntu install over thumb drive. it went through with display over serial cable, I have paused the installation though. Maybe I can even install Windows and overclock the CPU?

Before I do any changes I want to duplicate the ssd I will report back once I get more info.

I suspect I will be able to use it like a regular pc without an option to use any of the qsfp nic's. as the bios only show an option for built-in Intel one.

 

[alsenior]

If its running ONIE you could probably run SONIC on it. Worth trying the standard broadcom image to see if it detects the ASIC and the ports

 

[klui]

Worth a try but doubtful SONiC will work on it unless it's mentioned in its HCL. The software needs a SAI created that controls the switching ASIC through I2C and the abstraction interface module typically comes from the switch vendor... unless you're someone who knows switch hardware intimately like @okrasit who created one for Celestica's D4040.

 

[Bartosz Grzelak]

I think nokia wbx is a weird case, because if you had ability to run sonic or onl on it how can they justify charges for Nuage. anyhow got the Ubuntu running on it. it behaves like regular pc asic is listed in the lshw under Ethernet controllers. however is not listed under ip addr.

I shall try the sonic today and see. There is a guy who is selling Nuage software license SD card for a 500plus. Not sure If it's worth the risk. I am actually thinking about returning the switch to the seller.

 

[Bartosz Grzelak]

oh great now I think I messed up onie, it won't allow me to boot to it after recovery.

 

[darkfader]

oh great now I think I messed up onie, it won't allow me to boot to it after recovery.

as someone looking at that switch on Ebay and following your thread I have to say that the ending was really anti-climatic.
(I hope you can get ONIE back to respond, good luck!)

 

[Bartosz Grzelak]

@darkfader , Yeah, I ended up getting back ONIE. It turns out ONIE was booting on legacy bios setting. Ubuntu installation just switched the booting method to UEFI. Once I switched booting to legacy I was able to access the ONIE. I tried installing SONIC later on, it did install but no switch functions were available. This switch is behind contracts and its locked as much as possible. Not sure how recyclers get hold of them. Advise for potential buyers: You can buy those for cheap and use them as typical x86 server. They have one usb port at the front, intel D xeon and 16gb ddr4 ram, sata or nvme on board port, but switching wont be available. Oh and the switch can scream. I mean.. it can almost take off. I will be returning the switch, just need to book return with the courier. Here is the link to the listing I got the switch from : Nokia 210 WBX 32QSFP28 - 32 QSFP Ports - ONIE installed | eBay

 

[NablaSquaredG]

Yep, everything as expected.

Thank you for serving as a guinea pig!

 

[Bartosz Grzelak]

I kind of expected this could happen too.

Who needs 100gig at home anyway? I am not even an IT professional. I don't have any workloads that would require such capabilities. I am CGI artist by trade and I use my network to tie up all the render machines to combine compute and also for storage, that's all. I think I had to just fulfill my curiosity.

 

[darkfader]

Do you happen to know what switch chip it has?

And besides curiosity, it's also a waste if those things just end up as scrap. Gotta try.

It would be beautiful if Nokia/Nuage would have spotted this.
In an ideal world they would have supported you with a sensible discount SW offer..

 

[NablaSquaredG]

And besides curiosity, it's also a waste if those things just end up as scrap. Gotta try.

I've been told it's a BCM56960 (Broadcom Tomahawk 1, 3.2 Tbps)

 

[Stephan]

Compared to Arista, Mellanox, Brocade, these Nokias are niche-niche of tinkerers' niche. Not many are around. If this was open hardware with at least some 1000 pages of specs, people could at least try to port some network OS over. Is everything NDA with this model? If yes, then its formerly expensive scrap. Even die-hards like myself would imho not reverse engineer old binaries, simply because nobody has this switch or could buy one. Let alone shell out 15k for a software license to run stuff in the basement for family and fun.

 

[NablaSquaredG]

Is everything NDA with this model?

You can't even get the hardware manual without a support account (which requires a contact person at Nokia)...

Even die-hards like myself would imho not reverse engineer old binaries, simply because nobody has this switch or could buy one. Let alone shell out 15k for a software license to run stuff in the basement for family and fun.

Yeah exactly

Even if all specs were publicly available, I doubt that anyone would be willing to spend the significant amount of time required to fix SoNIC or another open NOS for this switch

 

[Rand__]

Who needs 100gig at home anyway?

*cough*
Now *need* is a strong word....

 

[tsteine]

*cough*
Now *need* is a strong word....

Yeah. I probably saturate 100gbit on a single port... 0.001% of the time (basically any time I run a benchmark for nvmeof against multiple drives)
Do you need it? Certainly not.. Is it fun to play around with though? Hell yes.

 

[Stephan]

Like they say over on r/homelab, "ruining the word humble since 2012". In the various switch threads on STH there is some pretty unique stuff to be found. So if anyone is eyeing 40..100 Gbps, they certainly came to the right place.

 

[Labs]

I've been told it's a BCM56960 (Broadcom Tomahawk 1, 3.2 Tbps)

Datasheet says 3.6Tbps for 48SFP28 6QSFP28 and 6.4Tbps for the 32QSFP28 model.

 

[Labs]

@Bartosz Grzelak Did you return the switch? Can I ask some questions if you remember? The switch only had ONIE? Or did it also boot to the Hypervisor (yes, this is how they call it).

These switches don't run bare metal like we are used to. They run the hypervisor which is based on CentOS 7.3 (for release 5.1) and on top you have the SROS VM (aka VSG VM). From Linux prompt you need to run "console vsgvm" to go to the switch CLI.

I guess (for sure) the hypervisor is doing pass-through of the Broadcom ASIC PCIe to this VM.

I guess the ONIE installer/updater also installs the hypervisor Linux part because the image quite big 500MB and the VSG VM (bof/both) is only 64MB in size. There are additional packages/VMs that can be added on top to have a nice integrated solution.

VSG VM is running on qemu-kvm so I guess you can run your own small VMs also.

I found this switch on eBay and I started to research it but I am not sure if I will buy one. I need to check how is with that lic part..
I would like to have a 25/100Gbe telco grade switch for lab and I don't see anything cheap on eBay. This one looks good for that price.
At the moment that switch is worth the price of the 16GB DDR4 SODIMM + 64GB/8GB/16GB SSD M.2 I saw different reports about internal SSD size, so with that said 50 Euro is the fair price for it.