Azure Sonic on the Arista 7050QX-32

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I finally got to spend some quality time with my Arista 7050QX-32 and Azure Sonic. Sonic, out of the box without any configuration whatsoever, has BGP running on all of the interfaces. This is intended to allow Layer 3 native IP networking for each of the hosts and their containers, significantly simplifying networking without the typical L2 container networking overlays. After installing and configuring my hosts for BGP, I can get TCP performance at line rate between containers on different hosts:

$ docker run -it --name mycentos centos bash
[root@1e24805a735c /]# yum install iperf

[root@1e24805a735c]# iperf -c
Client connecting to, TCP port 5001
TCP window size: 2.00 MByte (default)
[ 3] local port 34042 connected with port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 24.7 GBytes 21.2 Gbits/sec

[root@1e24805a735c]# iperf -c -P 4
Client connecting to, TCP port 5001
TCP window size: 2.00 MByte (default)
[ 6] local port 34050 connected with port 5001
[ 4] local port 34046 connected with port 5001
[ 3] local port 34044 connected with port 5001
[ 5] local port 34048 connected with port 5001
[ ID] Interval Transfer Bandwidth
[ 6] 0.0-10.0 sec 10.8 GBytes 9.27 Gbits/sec
[ 4] 0.0-10.0 sec 10.6 GBytes 9.12 Gbits/sec
[ 3] 0.0-10.0 sec 10.6 GBytes 9.13 Gbits/sec
[ 5] 0.0-10.0 sec 11.2 GBytes 9.60 Gbits/sec
[SUM] 0.0-10.0 sec 43.2 GBytes 37.1 Gbits/sec


L3 networking has the advantage of fewer proprietary extensions that enable large cloud-like fabrics while allowing dynamic config via standard routing software and isolating tenants. (that's the sales pitch anyways, in buzzword lingo)

I'll post/spam my experiences installing it while it's still fresh.

1. Get into aboot from a tty.

I used kermit:
(/home/admin/) C-Kermit>set line /dev/ttyS0
(/home/admin/) C-Kermit>set speed 9600
(/home/admin/) C-Kermit>conn

Boot the switch and hit ctrl-c at the prompt. You should have a about shell at this point.

2. Copy the Sonic distribution for Arista from here:
Supported Devices and Platforms · Azure/SONiC Wiki · GitHub

I put this on a USB stick and inserted into the switch. Aboot will mount the USB as /mnt/usb1 and the flash as /mnt/flash.

Backup the existing EOS software onto the USB, it will be erased by the Sonic install, there will be no backup saved on the switch.

# cd /mnt/flash
# cp -a ./* /mnt/usb1/
# cp /mnt/usb1/sonic-aboot-broadcom.swi .

3. Boot Sonic

# boot /mnt/flash/sonic-aboot-broadcom.swi

This erases the flash and installs Sonic onto it. You clear the flash again and boot EOS by following the aboot instructions here:
EOS Section 6.5: Aboot Shell - Arista

I used 'fullrecover' and 'boot EOS-*.swi' several times. The EOS, boot-config and startup-config can be restored from the USB drive.

4. Login to sonic:

login: admin
passwd: YourPaSsWoRd

The eth0 device is the gigabit management interface, it will start dhclient to get the address. I usually setup my /etc/dhcp/dhcpd.conf to hand out addresses based on the MAC address (use 'ip addr show eth0' in the tty):

host arista {
option host-name "arista.local";
hardware ethernet 44:4c:a8:12:87:c6;

After eth0 is configured, you can ssh admin@ and skip the slow tty.
5. Sonic does not dynamically adjust the fans on this switch. It puts them at high speed, so it will be very annoying if the switch is anywhere close to you. You can read the sensors and adjust the fans manually, so there must be some software out there that does dynamically adjust them. I throttle the fans manually.

# echo 120 > /sys/devices/platform/sb800-fans/hwmon/hwmon1/pwm1
# echo 120 > /sys/devices/platform/sb800-fans/hwmon/hwmon1/pwm2
# echo 120 > /sys/devices/platform/sb800-fans/hwmon/hwmon1/pwm3
# echo 120 > /sys/devices/platform/sb800-fans/hwmon/hwmon1/pwm4
# cat /sys/devices/platform/sb800-fans/hwmon/hwmon1/fan1_input

The command 'show' is the cli way of examining the state of the switch:

# show environment
Command: sudo sensors
Adapter: PCI adapter
Cpu temp sensor: +36.0 C (high = +70.0 C)

Adapter: ISA adapter
fan1: 12611 RPM
fan2: 12611 RPM
fan3: 12611 RPM
fan4: 12381 RPM

** Update:

There is a fancontrol daemon, but it needs configuration via pwmconfig. I think this needs to be done in the pmon container, at least that works for me.

# docker exec -it pmon bash
# pwmconfig

"pwmconfig" will allow you to choose which temp sensor devices control which fans. The 'fancontrol' daemon doesn't quite agree with "pwmconfig" for some temp sensors, so I used the CPU temperature to control the fan speed, since it is the highest value.

Test it:

# VERBOSE=1 /etc/init.d/fancontrol start

It should start automatically on boot by /usr/bin/ when it exists in the platform config /usr/share/sonic/device/x86_64-arista_7050_qx32/fancontrol

6. Exploring Sonic

The architecture of Sonic is described here:

Architecture · Azure/SONiC Wiki · GitHub

It is a collection of daemons which use a Redis database to communicate and persist configuration. You can use 'redis-cli' to examine the database and the activity of the daemons. For example, this shows what the daemons are doing:

# redis-cli psubscribe '*'

The file /etc/sonic/config_db.json contains the startup configuration. The Redis database is populated from this and the daemons configure the switch by using Redis operations. OpenSwitch also has foundations for a Redis based configuration, but it seems a bit behind, whereas Microsoft is building a new Sonic version daily (Artifacts of buildimage-brcm-aboot-all #274 : /target [Jenkins]).

The daemons in Sonic are inside docker containers. For example, the BGP container looks like this:

# docker ps -a
224f774178f0 docker-fpm-quagga:latest "/usr/bin/supervisord" 4 days ago Up 4 days bgp

# docker exec -it bgp bash
# ps ax
1 ? Ss+ 2:16 /usr/bin/python /usr/bin/supervisord
29 ? S 0:00 python /usr/bin/bgpcfgd
33 ? Sl 0:03 /usr/sbin/rsyslogd -n
38 ? S 0:02 /usr/lib/quagga/zebra -A
40 ? S 1:26 /usr/lib/quagga/bgpd -A -F
42 ? Sl 0:00 fpmsyncd
180 ? Ss 0:00 bash
184 ? R+ 0:00 ps ax

# cat /etc/quagga/bgpd.conf
router bgp 65100
bgp log-neighbor-changes
bgp bestpath as-path multipath-relax
no bgp default ipv4-unicast
bgp router-id
Last edited:
7. Setting up the IP addresses

As it is installed in build #270, Sonic defines each of the 32 ports with a with an EthernetX device and a 10.0.0.Y address, where X = (port-number-1) * 4 and Y = (port-number-1) * 2 (where port-number is 1 -> 32).

# ip addr show | grep inet
inet brd scope global Ethernet0
inet brd scope global Ethernet4
inet brd scope global Ethernet8

The address to use when you connect to these ports is 10.0.0.Z/31, where Z is (port-number-1) * 2 + 1. The first 16 ports use internal BGP, the second 16 ports are external BGP. Since external BGP is easier to configure than internal, I used the ports 17 -> 32 (I plan on creating a L2 lan with the first 16 ports anyway).

The addresses assigned to the hosts are

port 17 =
port 18 =
port 19 =

Since the layer 2 network set up using this config is confined to the link between the switch and the port, the host has to route through the switch in order to connect with the other ports otherwise any IP address other than the link pair will try to route to the wrong network. I did this by setting up my router as the default-gateway, which propagates default route through the switch to the hosts.

Install quagga and start zebra:
# yum install quagga
# systemctl start zebra

My 40gbe devices are named fo0 and fo1.

# vtysh
router# config term
router(config)# log file /var/log/quagga/quagga.log
router(config)# interface fo0
router(config-if)# ip address
router(config-if)# no shutdown
router(config-if)# end
router# write
Building Configuration...
Configuration saved to /etc/quagga/zebra.conf

You should be able to ping now, which is port 25 on the switch.
# ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=0.198 ms

One other thing I did on the router was route the range through the switch:

# ip route add via dev fo0

Repeat the above for each host connected, except for the last line.
8. Setting up BGP routing

Each of my hosts has a docker0 device on it using a unique ip address range. BGP is going to distribute the routes for the docker0 brige across the cluster.

# cat /etc/docker/daemon.json
"dns": [""],
"iptables": false,
"ip-forward": false,
"ip-masq": false,
"storage-driver": "btrfs",
"bip": "",
"fixed-cidr": "",
"mtu": 1500

The dns entry is a caching named on the router. I set up a loopback device with this ip address and distributed it with BGP.

Here is the /etc/quagga/bgpd.conf for the above docker bridge:

# cat /etc/quagga/bgpd.conf
hostname myhost
password zebra
log file /var/log/quagga/bgp.log

router bgp 64005
bgp router-id
neighbor remote-as 65100

Start bgpd
# systemctl start bgpd

The 64005 AS number above is unique for each port from 17 -> 32. These are configured be on the switch in the BGP container in /etc/quagga/bgpd.conf via the Redis database.

port 17 = 64001
port 18 = 64002
port 19 = 64003
port 21 = 64005

The above host is in port 21. After start bgpd on each of the hosts, you should see the routes populated by BGP.

On the switch:

root@arista:/# ip route show
default via dev Ethernet96 proto zebra
... via dev Ethernet80 proto zebra

All the hosts are set up the same, except for the router. The router has this in the /etc/quagga/bgpd.conf:

hostname router
password zebra
log file /var/log/quagga/bgp.log
router bgp 64009
bgp router-id
neighbor remote-as 65100
neighbor default-originate

Note the "default-originate" line. This propagates the default route to the hosts and the switch.

I also block the BGP port from the internet side:
# iptables -A INPUT -i te0 -p udp -m udp --dport 179 -j DROP
# iptables -A INPUT -i te0 -p tcp -m tcp --dport 179 -j DROP