Celestica Seastone DX010 - SONiC in L2 mode, need your help

[MichalPL]

Hello,

I bought some time ago Celestica DX010, and I need your help with L2 configuration. I have install SONiC on it - this was quite easy, switch looks like it's working (green lights when connected with 100GbE equipment, also after changing port speed to 40GbE it's looks like connected with Cisco 3064PQ or 3164Q), but it is in the L3 mode, and I don't know how to use it.

My goal is to use it in the L2 mode, or configure L3 to be working as L2 mode. No fancy options (maybe turning on autonegotiation will be good).

What I see at this moment if I plug it to Cisco 3064PQ it's interfering with my servers (they are on 10.0.0.1 and 10.0.0.10) and not forwarding any traffic.
here is what I see (port channels is my LACP tests):







Questions:
1. How to switch it to L2 ? is it a command or should I edit configuration files ?
2. If not possible (tips from here: Azure/SONiC but tweaked for Celestica are not working) how to configure L3 to be like L2 ? I mean to operate like $30 unmanaged switch but 100x faster. Just MAC table and ports ?

 

[MichalPL]

Ok, after OS update (to 615) it works! it's an amazing router for this price obv. (and it's similar to Cisco 3064PQ or 3164Q)

working ver:


The only issue is this version of sonic have problem with auto-negotiation (when done thru Broadcom commands it's not working too), so we fixed 12 ports for 40GbE, 16 for 100GbE, 4 for 80GbE LACP and 4 for 200 GbE LACP
anybody solve the this issue ?

 

[nasbdh9]

sudo sonic-cfggen -H -k Celestica-DX010 --preset=l2 > config_db.json

sudo config reload config_db.json -y

 

[MichalPL]

sudo sonic-cfggen -H -k Celestica-DX010 --preset=l2 > config_db.json

Thank you, generated much nicer and cleaner file that we were able to edit based on the original one

Do you know if it is possible to turn on autonegotiation on (most of the cards I am using are ConntectX 3 40GbE, I have few ConntectX 4 100GbE) it should be supported by platform, but seems to be not supported in sonic directly ?

 

[nasbdh9]

Thank you, generated much nicer and cleaner file that we were able to edit based on the original one

Do you know if it is possible to turn on autonegotiation on (most of the cards I am using are ConntectX 3 40GbE, I have few ConntectX 4 100GbE) it should be supported by platform, but seems to be not supported in sonic directly ?

The four channels of a port can only operate at the same rate, so auto-negotiation
This may be problem with Broadcom upstream, but I not sure

 

[MichalPL]

Yes it will be always 4 same lanes for each port - just dont want to fix them on QSFP+ or QSFP28. I don't have any SFP28 cards (some 10GbE SFP+ I can connect via Cisco modded 3064PQ or Cisco 3164Q connected via 80GbE LACP)

 

[secxsquared]

Yes it will be always 4 same lanes for each port - just dont want to fix them on QSFP+ or QSFP28. I don't have any SFP28 cards (some 10GbE SFP+ I can connect via Cisco modded 3064PQ or Cisco 3164Q connected via 80GbE LACP)

Try setting autoneg mode. See this Celestica Seastone DX-010 no-carrier on 100gig · Issue #384 · Azure/SONiC

 

[secxsquared]

Ok, after OS update (to 615) it works! it's an amazing router for this price obv. (and it's similar to Cisco 3064PQ or 3164Q)

working ver:
View attachment 17983

The only issue is this version of sonic have problem with auto-negotiation (when done thru Broadcom commands it's not working too), so we fixed 12 ports for 40GbE, 16 for 100GbE, 4 for 80GbE LACP and 4 for 200 GbE LACP
anybody solve the this issue ?

Supposedly they are shipping a new version with changes to autonegotiation at the end of June. Azure/SONiC

 

[im.thatoneguy]

Did the auto negotiation actually ship?

Also how's the volume on the Celestica? Hair Dryer or tolerable? Anyone tried venting the top and making it literally a "top of rack" unit with some big fans directly over the heat sinks?

 

[klui]

I just tried the 202106 branch built on Aug 11 that has autoneg but it doesn't work on the DX010. I set a port using RS FEC and links up. Once autoneg is enabled, it doesn't link up.

It's also buggy. Enable it and play around with the setting in trying to link with a 40G port and eventually the port won't link up even when autoneg is disabled unless you reboot the switch. Media type recognition is also problematic. The first time you plug in something is fine but when you unplug and replug into another port the status won't show it move to the current port. Breakouts aren't dynamic so autoneg only knows about 40G, 42G(?), and 100G. Even if autoneg is working you can't use 10G, 25G, 50G through breakouts without configuring the port then interrupting all port traffic. Dynamic breakout is scheduled for the 202111 branch.

The default fan setting sounds like a hair dryer on low inside a metallic chamber.

 

[im.thatoneguy]

sudo sonic-cfggen -H -k Celestica-DX010 --preset=l2 > config_db.json

sudo config reload config_db.json -y

Just tried this and now...

admin@sonic:~$ show platform fan
Fan Not detected
admin@sonic:~$ show platform temperature
Thermal Not detected

Looks like that preset may be broken.

 

[nasbdh9]

Just tried this and now...

admin@sonic:~$ show platform fan
Fan Not detected
admin@sonic:~$ show platform temperature
Thermal Not detected

Looks like that preset may be broken.

The L2 config automatically generated in the current version is incomplete, you need to manually copy some T1 config info

 

[im.thatoneguy]

sudo sonic-cfggen --preset l2 -p -H -k Seastone-DX010 >/etc/sonic/config_db.json

not sure what the -p flag does, but it does generate a proper config. Also "Seastone-DX010" not Celestica-DX010 for the sku.

 

[DCswitch]

On the Seastone DX010, I cannot figure out how to use the SFP+ port. Has anyone got this working?

 

[im.thatoneguy]

I don't think it's functional. I used a QSFP to SFP physical adapter and that worked when configured for split 40g. (And then plugged in a 10G baseT QSFP)

 

[Sjhwilkes]

Agreed, never been able to get the SFP+ port working either.

 

[klui]

SONiC doesn't support the SFP+ port. Neither does Cumulus.

 

[pimposh]

HPE Plexxi NOS supports it.