Does SONiC actually work for people?

[Scott Laird]

I'm trying to get SONiC working on an Edge-ware/Accton AS7326-56X, and *wow* this is non-trivial. Minor issues so far:

• Changing interface speed between 10 and 25 Gbps requires editing hardware definition files. This may only be true for my hardware; it looks like it's generally supposed to work. It took me ~2 hours to get a working connection between my old switch and my new (maybe?) one.
• There's no actual documentation on how to set up BGP. SONiC's docs suggest (but don't actually say) that editing /etc/sonic/config_db.json and running 'config reload' will do the trick. This does not appear to be the case; nothing from there ever appears in /etc/sonic/frr. Other docs suggest that manually running vtysh and configuring FRR directly is best. This does not work either, as SONiC overwrites FRR's configs all the time. The best process seems to be to disable this overwriting (via adding "docker_routing_config_mode": "split" to "DEVICE_METADATA" in config_db.json and restarting the switch) and then just use FRR/Quagga directly.
• Except that doesn't seem to work, either. After a couple hours, it looks like SONiC isn't actually configured to actually run BGP at all. After octuple-checking my (very minimal) configs, it looks like SONiC never actually tries to open a BGP connection to my Juniper switch, and whenever Juniper tries to connect to it, it *immediately* sends a FIN right after SYN/SYN-ACK/ACK. Without logging anything anywhere. I'm basically just using an already-working config and changing IP addresses. This isn't supposed to be the hard part of BGP.
• There's some suggestion that BGP broke in the master branch in March and no one's fixed it yet. Surely I'm not running a daily build, right? Check, check... yeah. I am. The download links from the SONiC download page fetch daily builds, not the most recent stable branch. (Scroll the table sideways; the download links aren't actually visible by default).

So far, I'm not feeling very impressed. It's nice having a switch that runs Debian under the hood, and where tcpdump actually works, but there are a lot of features missing that would be really nice to have (OSPF! RSTP!), and the *total lack of actual documentation* is painful. It's probably a great base to work with if you're rolling out multiple DCs and need a ton of switches that you're planning on controlling centrally, but the learning curve for a single switch is steep.

I'm going to try rolling back to the 201911 release and see if that fixes anything.

 

[Jeggs101]

I think it's like one of those projects where if you've got big infrastructure and a team it's great but if you don't, then you're running without a lot of the tooling