REALLY "sluggish" video drawing on X10SRL-F via IPMI VGA

Discussion in 'Processors and Motherboards' started by ljwobker, Jan 25, 2019.

  1. ljwobker

    ljwobker New Member

    Joined:
    Oct 12, 2017
    Messages:
    21
    Likes Received:
    4
    I've put together a Linux/KVM system based around an X10SRL-F motherboard, and I'm stumped on what I can only describe as "extremely slow or sluggish video". What I mean by this is that when the system boots, the console messages that appear on the screen scroll excruciatingly slowly, and this adds something like 90-120 seconds to the boot process. My background is in routers and switches that use serial consoles, and the only way I can describe it is like watching a 2400 baud terminal scroll up and down -- it's so slow that you can almost read every line as it appears. Once the machine boots into the graphical desktop, the video is STILL very sluggish, you can visually see that it takes a long time to redraw or resize windows or paint the initial screens of any application that loads.

    I don't *think* it's related to the boot/console parameters in any way, because it behaves the same way regardless of whether it's booting (i.e. "console") or already in the graphical desktop.

    This is my first IPMI capable board, so I'm pretty sure there is something in between the main system and the IMPI video that's slowing things down, but I don't have the first idea where to look for this. I couldn't come up with a better way to describe in words the difference between the "fast" video and the "slow" video, so I just recorded a couple of short video clips and put them up on google drive: it's very obvious by watching and comparing them how much difference there is in how fast the video is drawn.

    X10SRL-F video - Google Drive

    Interestingly enough, once I get the system booted, I can do a kexec-reboot (error10/kexec-reboot) and once the system comes back via that approach, it's WAY more responsive. I can't figure out if there's some kind of driver/accelerator that doesn't get loaded the first time around or something like that? I could almost convince myself that if it needed to get booted into the desktop to load the driver this would make sense, but even the first shutdown (i.e. once you've gotten into the Linux desktop) it's still very slow.
    To break it down step by step with speeds, it's something like this:

    1. cold boot the machine. IPMI, BIOS, all that stuff operates at normal speed/responsiveness.
    2. once the main OS (Ubuntu 18.04.1) starts loading, the boot console messages get very sluggish/laggy and the system crawls along through the boot process. It's important for me to note that the video is synchronous, it's NOT like the system is booting normally and the video is just dropping a bunch of frames along the way... because the video is so slow it bogs down the entire boot process -see logs below... (google drive video link: you can really see the impact at about the 25 second point)
    3. The main desktop loads, but the video is still super laggy and windows take forever to paint. (video)
    4. if I do a regular reboot (or shutdown) the behavior stays the same... even the shutdown console is still really slow (video)
    5. BUT, if I do a kexec-reboot (bounce just the kernel) then it comes back a lot faster. (video after kexec-reboot)
    6. after the kexec-reboot when the desktop loads, it's quite responsive and snappy! (video)

    On this machine, one of the last messages to hit syslog as the system boots is the various ports in the bridge going into forwarding state... you can see there's almost a three minute difference between when the video is "good" and things move quickly compared to when it's "bad" and they move really slowly...


    Jan 25 12:02:12 lwobker-vms kernel: [ 15.804309] bridge1: port 1(eno1) entered forwarding state
    Jan 25 12:02:16 lwobker-vms kernel: [ 19.868160] bridge1: port 2(ens2) entered forwarding state
    Jan 25 12:02:18 lwobker-vms kernel: [ 21.640799] bridge1: port 3(vnet0) entered forwarding state

    Jan 25 12:03:16 lwobker-vms kernel: [ 16.784198] bridge1: port 1(eno1) entered forwarding state
    Jan 25 12:03:19 lwobker-vms kernel: [ 20.284144] bridge1: port 2(ens2) entered forwarding state
    Jan 25 12:03:21 lwobker-vms kernel: [ 22.265671] bridge1: port 3(vnet0) entered forwarding state

    Jan 25 14:52:47 lwobker-vms kernel: [ 12.746976] bridge1: port 1(eno1) entered forwarding state
    Jan 25 14:52:53 lwobker-vms kernel: [ 19.062700] bridge1: port 3(vnet0) entered forwarding state
    Jan 25 14:52:54 lwobker-vms kernel: [ 19.900282] bridge1: port 2(ens2) entered forwarding state




    Jan 25 14:28:08 lwobker-vms kernel: [ 154.754752] bridge1: port 1(eno1) entered forwarding state
    Jan 25 14:28:08 lwobker-vms kernel: [ 160.380175] bridge1: port 2(ens2) entered forwarding state
    Jan 25 14:28:39 lwobker-vms kernel: [ 193.922273] bridge1: port 3(vnet0) entered forwarding state

    Jan 25 14:36:09 lwobker-vms kernel: [ 153.044693] bridge1: port 1(eno1) entered forwarding state
    Jan 25 14:36:09 lwobker-vms kernel: [ 158.652181] bridge1: port 2(ens2) entered forwarding state
    Jan 25 14:36:40 lwobker-vms kernel: [ 194.898956] bridge1: port 3(vnet0) entered forwarding state

    Jan 25 14:49:21 lwobker-vms kernel: [ 146.123308] bridge1: port 1(eno1) entered forwarding state
    Jan 25 14:49:21 lwobker-vms kernel: [ 152.252138] bridge1: port 2(ens2) entered forwarding state
    Jan 25 14:49:44 lwobker-vms kernel: [ 187.832669] bridge1: port 3(vnet0) entered forwarding state


    Help!?!?!
     
    #1
  2. dandanio

    dandanio Member

    Joined:
    Oct 10, 2017
    Messages:
    64
    Likes Received:
    23
    I am not surprised at all. Not sure where you got the idea of using a KVM/ipmi for anything close to production, even worse in a graphical environment. Switch to a regular video and you will have great performance.
     
    #2
  3. RageBone

    RageBone Active Member

    Joined:
    Jul 11, 2017
    Messages:
    266
    Likes Received:
    70
    @ljwobker did you update the ipmi fw to the newest version?

    Is it the same if you aren't watching over iKVM? Boottimes whise I mean.
     
    #3
  4. ljwobker

    ljwobker New Member

    Joined:
    Oct 12, 2017
    Messages:
    21
    Likes Received:
    4
    So I never said it needed to be super fast, nor would I plan on doing anything like this in a "real" production environment. I was just curious as to whether this was a known / expected thing and/or if it could be mitigated. And I wouldn't consider it a graphical environment, it's just the boot console (which I believe in modern linux distros is usually actually graphical, but I think you get the point?)
     
    #4
  5. ljwobker

    ljwobker New Member

    Joined:
    Oct 12, 2017
    Messages:
    21
    Likes Received:
    4
    ARGH. Upgrading the IPMI firmware fixed it. I was told by the person I bought the board from that it had been updated to the latest, but I never actually verified this. Sorry for the spam ;-)
     
    #5
    RageBone likes this.
  6. EffrafaxOfWug

    EffrafaxOfWug Radioactive Member

    Joined:
    Feb 12, 2015
    Messages:
    1,079
    Likes Received:
    356
    T'was at least an interesting read for me - and an interesting failure mode (although I didn't watch the videos). I've never heard of a failure mode like this that actually apparently blocked the OS from functioning properly.

    Out of curiosity, do you know which IPMI version you running that suffered from this problem before you upgraded?
     
    #6
  7. ljwobker

    ljwobker New Member

    Joined:
    Oct 12, 2017
    Messages:
    21
    Likes Received:
    4
    Yeah it was really odd, like there was some horrible block/sync call in the frame buffer. As far as the IPMI versions, I think from something like 3.27 up to 3.65.
     
    #7
Similar Threads: REALLY sluggish
Forum Title Date
Processors and Motherboards Change to 2011-3 platform is it really worth it Dec 2, 2019
Processors and Motherboards Do I really need the plastic carrier for a e5-26xx Aug 10, 2019
Processors and Motherboards Supermicro X9DRX - do I really need NVMe BIOS mod? Nov 23, 2018
Processors and Motherboards IPMI help - Really frustrated Sep 18, 2018
Processors and Motherboards Power consumption at idle - really any benefit? Mar 13, 2017

Share This Page