Celestica D4040

Discussion in 'Networking' started by busterswt, Apr 20, 2019.

  1. busterswt

    busterswt Member

    Joined:
    Apr 30, 2017
    Messages:
    52
    Likes Received:
    11
    Any Celestica owners out there? Curious to know what you’re running - whether it’s ICOS or Cumulus or something else.

    Cheers!
    JD
     
    #1
  2. pcmoore

    pcmoore Member

    Joined:
    Apr 14, 2018
    Messages:
    97
    Likes Received:
    20
    I've been curious about these, but there doesn't seem to be a lot of folks using them (or rather those that do are very quiet). Do you have one? If so, what are you using on it?
     
    #2
  3. busterswt

    busterswt Member

    Joined:
    Apr 30, 2017
    Messages:
    52
    Likes Received:
    11
    That’s a good question. Mine came without storage installed, so I had to go thru the process of building ONIE and finding a NOS to install. The original Ubuntu/ICOS is unobtainable, so I went with Cumulus. I’ve got a trial license, but switchd won’t start due to an eeprom or some other issue. I need to be able to compare some output to know for sure.

    Long story short, I can’t tell you much about it, but I know there are some STH members that have one.
     
    #3
  4. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,579
    Likes Received:
    542
    I do have two Celestica's but not 4040s.
    One runs Sonic the other a trial of OCNos.
    Have not managed to get a trial for cumulus or would have given it a whirl.
     
    #4
  5. Labs

    Labs New Member

    Joined:
    Mar 21, 2019
    Messages:
    25
    Likes Received:
    4
    I played with a D4040 some months ago and it was running ICOS with L2/L3/Routing/IPv6/etc enabled. Not sure if all come the same way.
    I also checked the breakout cables 40G QSFP to 4x10G SFP+ from IBM and it was OK. Different QSFP-QSFP passive cables from different vendors also were OK.

    SSD was mSATA 16GB and it had 4GB DDR3L I think ECC unbuffered and a empty slot for addition module. Both SSD and RAM module were with innodisk label.

    What I noticed was the CPU is the Intel C2xyz model which is the affected one same like in Cisco ASA and some ISR routers.
    I checked at that time the specific model and stepping to confirm the unit I had was affected and it was.

    I don't have the switch anymore so I cannot help with more details...
     
    #5
    busterswt likes this.
  6. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,385
    Likes Received:
    1,115
    Interestingly, the bug shouldn't affect the D4040 switches. The c2xxx failure mode is that the clock for the LPC bus fails. The c2 series can boot from two places, either a bootrom connected via the LPC bus, or via SPI flash. When the LPC clock fails, it will no longer be able to read a bootrom over the LPC bus next time you try to boot it, so it will be bricked. However for devices that boot off SPI connected flash, it won't effect functionality. That's why some vendors had massive failure rates from this bug (like cisco who's ASA's booted off LPC connected bootroms), and others had pretty much none

    the D4040 boots off a pair of winbond W25Q64FV 64mbit SPI flash chips. I believe the two SPI chips are arranged in a redundant bootloader configuration following the intel recommended method for the C2000 series: https://www.intel.com/content/dam/w...ant-spi-flash-with-failover-boot-app-note.pdf
     
    #6
    Labs likes this.
  7. Labs

    Labs New Member

    Joined:
    Mar 21, 2019
    Messages:
    25
    Likes Received:
    4
    Thanks for the explanation. I was looking for this for some time. So it is somehow safe to say that all switches based on Intel C2000 CPUs that boot from SPI flash are not affected by this bug. I remember there is also one Arista switch based on some Intel CPU but I don't remember the exact model. The rest of Arista switches are based on AMD from what I saw.

    Thanks again!
     
    #7
  8. okrasit

    okrasit New Member

    Joined:
    Jun 28, 2019
    Messages:
    16
    Likes Received:
    18
    I have a D4040 running sonic. I've got the support done almost completely, just the leds and qsfp hotplugging in the works.:rolleyes:
     
    #8
    fohdeesha likes this.
  9. okrasit

    okrasit New Member

    Joined:
    Jun 28, 2019
    Messages:
    16
    Likes Received:
    18
    AFAIK the CPLDs on the D4040 sit on the LPC bus. If the bus were to fail, it'd probably brick the device. If there's any one location, in the boot code, that waits for a bit to be set/cleared, eg. polling for a status change of a register, it'd hang there forever (no boot). I think, this is what's happening with most of the "failing" devices. It's not that the boot rom isn't accessible but, the bootcode/bios poking the LPC-bus, causing the code to hang. :rolleyes:

    There was a mention of VDDIO 1.8V being "safe" for the E3xxx atoms, somewhere. That might also apply for the 2xxx series, if they even can do LPC @1.8V. :oops:
     
    #9
    Labs likes this.
  10. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,385
    Likes Received:
    1,115
    Are they really? I thought the MachXO2's in this thing were on the PCI bus, but I haven't had a chance to dive into the D4040 yet. if they're on the LPC bus and it fails then that'd certainly cause issues
     
    #10
  11. okrasit

    okrasit New Member

    Joined:
    Jun 28, 2019
    Messages:
    16
    Likes Received:
    18
    They sure are. I think i'm going to add a pull-up resistor there, before the unit breaks down.
     
    #11
    Labs and fohdeesha like this.
  12. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,385
    Likes Received:
    1,115
    if you find the required LPC pins broken out to test pads anywhere to add a resistor post some pics!
     
    #12
  13. okrasit

    okrasit New Member

    Joined:
    Jun 28, 2019
    Messages:
    16
    Likes Received:
    18
    IMG_0687.JPG
    IMG_0689.JPG
    eyedrop.png
     
    #13
    Last edited: Jul 1, 2019
    klui, Labs and fohdeesha like this.
  14. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,385
    Likes Received:
    1,115
    beautiful!
     
    #14
    okrasit likes this.
  15. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    2,855
    Likes Received:
    427
    I admire people who can figure stuff like this out and just do it :)
    Way beyond my means or knowledge.
     
    #15
    klui and okrasit like this.
  16. Labs

    Labs New Member

    Joined:
    Mar 21, 2019
    Messages:
    25
    Likes Received:
    4
    #16
    okrasit likes this.
  17. okrasit

    okrasit New Member

    Joined:
    Jun 28, 2019
    Messages:
    16
    Likes Received:
    18
    Here's a dead d4040, with no output from serial console:
    SDS00003.png

    The same unit with a pull-up of 121Ohm, working:
    SDS00006.png
     
    #17
    fohdeesha and Labs like this.
  18. klui

    klui Member

    Joined:
    Feb 3, 2019
    Messages:
    37
    Likes Received:
    16
    I'm not familiar with electronic design. Did you put a 121 Ohm resistor in series with the capacitor at C410 and R562 where the resistor is covered in black tape?
     
    #18
  19. okrasit

    okrasit New Member

    Joined:
    Jun 28, 2019
    Messages:
    16
    Likes Received:
    18
    The right pad of R562 is the clock signal and the top pad of the C410 is just a 3.3V supply voltage. Yes, the black blob is the resistor. I had only smd resistors at hand, so there's a small pcb inside there, with the resistor soldered on.
    :rolleyes:
     
    #19
    Labs likes this.
  20. Labs

    Labs New Member

    Joined:
    Mar 21, 2019
    Messages:
    25
    Likes Received:
    4
    From what I read on different forums this fix is only to prolong the life of the unit because at some point they will still fail and they cannot be repaired anymore.

    Is it true or it can be considered a permanent fix?
     
    #20
Similar Threads: Celestica D4040
Forum Title Date
Networking Why is a Celestica switch listed in an Arista Mib? Jun 17, 2019

Share This Page