Modding/upgrading Arista switches?

Discussion in 'Networking' started by oddball, May 18, 2018.

  1. zxv

    zxv The more I C, the less I see.

    Joined:
    Sep 10, 2017
    Messages:
    89
    Likes Received:
    30
    Code:
    Component       Version
    --------------- ----------------------------
    Aboot           Aboot-norcal2-2.0.10-1458058
    
    Code:
    show version
    Arista DCS-7050QX-32-R
    Hardware version:    02.11
    
    This was purchased from Unixplus a couple of months ago.
     
    #61
    fohdeesha likes this.
  2. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,045
    Likes Received:
    757
    ah, looks like you have the non-S management board (raven if I remember right, norcal2)
     
    #62
  3. WANg

    WANg Active Member

    Joined:
    Jun 10, 2018
    Messages:
    426
    Likes Received:
    154
    localhost#show version detail | grep -i Aboot-norcal4

    Aboot Aboot-norcal4-4.1.0-3037060
     
    #63
  4. spali

    spali New Member

    Joined:
    Nov 4, 2018
    Messages:
    22
    Likes Received:
    2
    Just got mine today :D
    After handcrafting an adapter between a RJ45 patchcable and a USBserial adapter I just updated to 4.21.0F.

    directly after the update:
    Code:
    localhost#show version detail | grep -i Aboot-norcal
    Aboot           Aboot-norcal4-4.0.4-2086886
    
    PS: mine seems to be the same hardware revision, also no SATA and the rest looks the same

    which version of EOS do you have? I'm couros, because I know there is a 4.21.1F, but didn't find it somewhere.
     
    #64
  5. WANg

    WANg Active Member

    Joined:
    Jun 10, 2018
    Messages:
    426
    Likes Received:
    154
    EOS-4.20.2.1F
     
    #65
  6. WANg

    WANg Active Member

    Joined:
    Jun 10, 2018
    Messages:
    426
    Likes Received:
    154
    Okay, more fun with the 7050qx-32s - how to get around that stupid USB-DOM requirement.

    Managed to get one of those standard 9 pin USB-pinout-to-port adapters from Amazon - however, realized that the pinout was mirrored.

    If you look at Cactus Technology's super-helpful USB-DOM tech guide, they have a rather useful pinout right here:

    [​IMG]
    Note that for the 9 pin connector only pins 1, 3 5 and 7 are needed (9 is keyed out as an anti-Murphy measure). Note that due to the way how the 9 pin connector is wired, you'll have a vertically mirrored image (where all the needed pins are on 2/4/6/8 intead of 1/3/5/7).

    So what's the solution?

    Offset it by a row so all the pins line up properly (sorry, not sure why embedding the photo failed).

    I might dig the connectors out and re-terminate it later...or next time, buy and use one of these 4-pin-to-USB port adapters on Amazon instead and save the minor hassle. If you use that $4 adapter on Amazon you'll not need to buy a USB-DOM...it'll just be a simple USB thumbdrive instead. I'll recommend something small form-factor, like a Sandisk Cruzer Fit. Adapter + 16GB Cruzer fit is around $15 on Amazon prime 2-day shipping.
     
    #66
    Last edited: Nov 22, 2018
    fohdeesha likes this.
  7. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,045
    Likes Received:
    757
    latest is EOS-4.21.2F - sadly (at least in my OCD opinion) the EOS images do not upgrade/touch the aboot environment. I can understand why, more risky than flashing just fw and generally no need to. did find a hidden utility to flash a new coreboot/aboot bootloader, just need a source file. just grabbed aristas source code package, going to compile the latest coreboot + aboot with more options to boot other stuff easier (not needing to write to SPI).

    I also have norcal4-4.1.0-3037060, looks like that's the latest out of our samples so far
     
    #67
  8. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,533
    Likes Received:
    355
    What's the advantage of having a more recent version of aboot?
     
    #68
  9. spali

    spali New Member

    Joined:
    Nov 4, 2018
    Messages:
    22
    Likes Received:
    2
    I think if you just use the switch as intended then you won't have any benefits. But courios too what @fohdeesha is planning :)
     
    #69
  10. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,045
    Likes Received:
    757
    yeah, nothing for normal users (except maybe some really niche physical security/boot security issues patched)

    also here's a quick guide to properly format a new USB DOM so it appears like they do from factory. I originally just created a partition table on it, created sda1, then it would boot no problem, but kept getting weird "can't mount /dev/sda, /dev/sda1 already mounted" errors - so after looking through the aboot init scripts, it mounts the block device /dev/sda directly, no partitions, as vfat

    So on a new USB DOM/internal USB storage:

    Code:
    #unplug front thumb drive
    #power on
    umount /mnt/flash
    mkfs.vfat /dev/sda
    reboot
    
    #plug in front thumb drive
    #copy the main image
    cp /mnt/usb1/EOS-4.21.2F.swi /mnt/flash
    #copy configs over from your thumb drive if they exist
    cp /mnt/usb1/startup-config /mnt/flash
    cp /mnt/usb1/zerotouch-config /mnt/flash
    boot flash:EOS-4.21.2F.swi
    
    #it will boot into EOS
    #tell it to boot from internal flash automatically from now on
    enable
    config t
    boot system flash:EOS-4.21.2F.swi
    write
    maybe obvious to most of you, not sure. Most embedded usb booting devices like this I've used in the past mounted a partition, so wasn't used to it wanting the raw black device

    edit: neat, for aboot to boot the full SWI EOS image, it just kexecs the kernel from it, so coreboot does not get called again:
    aboot > EOS boot stage 0
    aboot >EOS boot stage 1
     
    #70
    Last edited: Nov 22, 2018
  11. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,045
    Likes Received:
    757
    also found a neat script that should speed up boot quite a lot, for people with onboard sata DOM or M2 flash.

    instead of saving space by storing only the compressed SWI with a yet again compressed squashfs filesystem inside it, which must be decompressed, copied and mounted to boot the switch every time (which is what takes 80% of the boot time), there's /lib/image-install script in aboot - it takes an SWI image and extracts it to a directory of your choosing (like a directory on your fast m2 flash) where it permanently lives, now on each boot it just mounts that already extracted file system directly so no uncompressing stage. the script even updates /mnt/flash/boot-config for you so boots in the future automatically boot from your new dir:

    Code:
    sed -i -e "/^SWI=/ cSWI=dir:${target}" /mnt/flash/boot-config
    it won't work with the onboard USB DOM, as the script requires an ext4 filesystem. You're welcome to attempt formatting the USB DOM as ext4, but aboot is hardcoded to mount it as vfat no matter what you do. Seems the only thing it's hardcoded to mount as ext4 is M2 flash or SATA DOMs.

    also it seems the "dir" boot location argument is undocumented, the manual only mentions the usual options of flash, nfs, etc, all of which look for an SWI file. To install an SWI from a usb thumb drive to an extracted & uncompressed directory on an M2 drive that you have formatted to ext4:

    Code:
    /lib/image-install -d /mnt/drive /mnt/usb1/EOS-4.21.2F.swi
    boot
    you should then be able to boot much faster, since it's now booting EOS off of proper non-USB flash, and the image is already decompressed. I would say you could then pull out the USB DOM entirely and just boot off reliable M2 flash, but it seems it will still store your switch config and boot config on /mnt/flash (the usb dom). probably a way to change that as well I haven't found but I've already been up way too late

    an install like this would also make it WAY easier to modify every bit of EOS to your liking: usually it's a read-only squashFS filesystem with just a couple persistent folders, but since this script unsquashes the FS to a static dir, you can edit away
     
    #71
    Last edited: Nov 22, 2018
    spali likes this.
  12. spali

    spali New Member

    Joined:
    Nov 4, 2018
    Messages:
    22
    Likes Received:
    2
    Anyway thinking about if it is worth to replace my USB-DOM. I thought about just put the swi files on the M.2. and then just change the boot_config to load it from there. And as you suggest, this could be optimized by already extracting in beforehand.

    Not tried yet, but if I'm right, then on every update, I would just need to image-install the swi to the disk (could be even a subdirectory!?) and point in boot-config to it. For me, this sounds like almost the same procedure as you would normally update the EOS.

    according to the docs you would replace:
    Code:
    copy usb1:/sourcefile flash:/destfile
    with
    Code:
    /lib/image-install -d /mnt/drive/4.21.2F /mnt/usb1/EOS-4.21.2F.swi
    not sure if this can be done in the booted EOS instead of aboot.

    and then replace
    Code:
    boot system flash:/EOS-4.13.2.swi
    with
    Code:
    boot system drive:/EOS-4.13.2
    or similar?

    Should work in general.
    Can anyone think of any caveat on not using this? I'm still to new to know what the impact could be.
     
    #72
  13. spali

    spali New Member

    Joined:
    Nov 4, 2018
    Messages:
    22
    Likes Received:
    2
    in booted EOS there the script available in /usr/sbin/image-install
    so
    Code:
    bash /usr/sbin/image-install -d /mnt/drive/4.21.2F /mnt/flash/EOS-4.21.2F.swi
    should do the trick.
    Just can't test it now because I don't have the M.2 yet.
     
    #73
  14. NathanM3

    NathanM3 New Member

    Joined:
    Dec 4, 2016
    Messages:
    2
    Likes Received:
    0
    Code:
    Arista DCS-7050QX-32S-F
    Hardware version:    21.32
    
    Software image version: 4.20.2.1F
    Architecture:           i386
    Internal build version: 4.20.2.1F-7225656.42021F
    
    Aboot           Aboot-norcal4-4.1.0-3037060
    
    Is the M.2 Sata or NVME?
     
    #74
  15. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,045
    Likes Received:
    757
    yup - it actually does it to a subdirectory named EOS by default (so if you point it to /mnt/drive it installs in /mnt/drive/EOS). the last line in the script automatically updates boot-config as well, so you shouldn't need to do that "boot system" command. if you did for some reason, the syntax is dir not drive, so something like boot system dir:/mnt/drive/EOS

    only caveat I can think of is it still stores your switch startup-config file on the USB DOM, so you can't get rid of it. I guess this is no different than a normal install, but I'm going to poke around and see if I can get it putting everything on m2 automatically so I can remove the USB part completely and only have to rely on one flash unit
     
    #75
    Last edited: Nov 22, 2018
  16. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,045
    Likes Received:
    757
    sata
     
    #76
  17. spali

    spali New Member

    Joined:
    Nov 4, 2018
    Messages:
    22
    Likes Received:
    2
    no was by accident, repaced in in hurry ;)
    I don't care the config on the USB-DOM, because it shouldn't impact the performance to much just for the configuration. At least this was my intention and you can keep an swi file there for the case the M.2 dies, then you just need to update the boot config to get the switch up again with the same config. For me the USB-DOM is just a pain because of the limited size, so if I just move the swi (extracted or not) to the M.2, I'm not limited anymore. Would be cool if the boot config would support a kind of fallback if the extracted (on M.2) fails it would fall back to the swi on the USB-DOM. So you could have a upgrade path like copy to flash (as documented) and extract from there with the script and somehow edit the boot-config to first look at the M.2 and as fallback take the SUB-DOM.
     
    #77
  18. WANg

    WANg Active Member

    Joined:
    Jun 10, 2018
    Messages:
    426
    Likes Received:
    154
    Well, assuming that boot system drive:/EOS_4.3.swi works. I thought aboot or EOS config can only take usb1: or flash: as its boot device designator. Oh well, my Crucial MX500 might be coming on Saturday, so we can give it the old college try on Monday after work (I am tuning the switch to be as quiet as possible before it comes anywhere close to my apartment).
     
    #78
  19. WANg

    WANg Active Member

    Joined:
    Jun 10, 2018
    Messages:
    426
    Likes Received:
    154
    Hm...I kinda doubt it - it looks like the powerucd profiles suggests that some Crow boards uses 1.5v DIMMs, while others uses the 1.35v DDR3L, but I am not sure the ECC RAM used by the Crows actually come in the 1.5v form, so while it is possible, it's fairly unlikely.

    There are 2 versions of the Clearlake switch - a Clearlake (12MB packet buffer) and a ClearLakePlus (16MB packet buffer). I spent the evening unpacking the squashfs image so I can poke around without firing up the switch (which is in my $dayjob office going through a burn-in anyways).

    So, fun times deciphering the entire 7050 family, here's the 411:

    7050 (model number)
    Connectivity type:
    TX (10GBaseT/copper connectivity - I have no idea why anyone thinks that this is a good idea)
    SX (10G SFP+ connectivity - much more like it)
    QX (40GbE QSFP+ connectivity - can be broken out into quadruple SFP+ links)

    If the Connectivity type is trailed with a 2, it's a "plus" model with more system RAM and a bigger packet buffer in the switch. The trailing S tells you whether you are getting 4 more SX ports on the switch or not. There is also a Q in certain models, but I think that's typically used by the SX models to denote the presence of 4 QX ports.

    Example: The DCS-7050TX72Q will have, hmmm...48 10GBaseT copper ports and 6 40 GbE QSFP+ ports. That's a Crow-only model, BTW.

    I'll come by tomorrow and drop the codenames for the switches, but they are a headache to decipher. I also found out that the CPU board model in a given switch is not defined by model (well, it is), but it could also be production date. It seems that there are 7050QX-32 and 32S models that can be both Raven and Crow, it only depend on whether it came out before or after the Turion II Neo CPUs are retired or not - if it's produced after the cutout it'll be the Crow (with the GX420CA). There are also some interesting information about the family of CPUs used by Arista in their hardware.
     
    #79
  20. spali

    spali New Member

    Joined:
    Nov 4, 2018
    Messages:
    22
    Likes Received:
    2
    I'm an idiot.... ordered an M-key only (NVMe)... had to send it back today.... did just not recognized it :) I think I'm ordering now too, a MX500.
    Checked mine after reading this, cpuinfo states GX-420CA.
    Now I'm totally confused what kind of RAM I need to look for my 32S. But not only your fault ;) I found a pin-compatible old 1GB RAM module (SU3U1333B1G9-B) which I gave a shot... it worked... except it could not boot fine because during boot of EOS oom-killer killed all new processes :). But at least it was detected (but not as 1GB):
    Code:
    System RAM: 746104 kB
    in the console.
    And this module is (according to memoty4less.com) 1.5V non-ECC. So it seems that the switch isn't that picky?
    @oddball mentioned in another thread what he had in his:
    I have TRF7251U67G1600G8-NYCBP and SU3U1333B1G9-B was also detected. So my conclusion: 1.35V and 1.5V seems to work in mine and also ECC and non ECC.
    I'm not a memory export, can't say what's a must for compatibility and what not... the only thing I know is ECC or Non-ECC should at least in theory not matter, what I proved on my side.

    So I have following questions:
    • 1.35V vs 1.5V compatibility?
    • Single Rank vs Dual Rank compatibility?
    • Buffered vs Unbuffered compatibility?
    • Registered vs Unregister compatibility (I think not compatible)?
    The problem of finding 16GB ECC for the 32S seems to be most because of Unregistered and Single/Dual Rank
     
    #80
Similar Threads: Modding/upgrading Arista
Forum Title Date
Networking Arista 7050QX - Cut-through - Store and Forward - iSCSI Feb 8, 2019
Networking Arista - disable STP on interface/port Jan 10, 2019
Networking Arista 7050QX - hard reboot and packet-storm Sep 20, 2018
Networking Arista DCS-7050QX-32S, 4-SFP+ ports - can't configure Sep 15, 2018
Networking Arista EOS for MLAG Upgrade Sep 11, 2018

Share This Page